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Executive Summary 

Urban Alliance, headquartered in Washington, DC, serves at- risk youth through its high school internship 
program, which provides training, mentoring, and work experience to high school seniors from distressed 
communities in Washington, DC; Baltimore; Northern Virginia; and Chicago. The program serves youth 
before they become disconnected, helping them successfully transition to higher education or 
employment after graduation. 

Urban Alliance has commissioned the Urban Institute to conduct a six-year, randomized controlled 
trial impact and process evaluation of its high school internship program. This report provides a process 
analysis of the program; the analysis is informed by extensive evaluator observation and interviews with 
staff, stakeholders, and youth. It also presents baseline information about Urban Alliance and the youth 
participating in its high school internship program in Washington, DC, and Baltimore in the 2011- 12 and 
2012- 13 program years. Subsequent reports as part of the impact study will describe the early- adulthood 
impacts of the Urban Alliance internship program on the youth it serves. Below is a summary of the 
findings in this first of three reports. 

Program Model 

• Goals for youth: The program's goals for youth are that they ( 1) improve their hard and soft job skills, 
(2) gain long-term, office- based employment experience, (3) graduate from high school, (4) attend 
college or a training program, and (5) identify long-term employment opportunities. 

• Program components: The program's key elements are ( 1) a paid internship in an office setting at a 
nonprofit organization, corporation, or government agency (daily after school and full-time during 
the summer); (2) soft and hard skills job training for 4- 6 weeks after school at the start of the 
program ("pre- work training") and Fridays after school thereafter ("workshops"); (3) coaching and 
mentoring provided by Urban Alliance Program Coordinators and job mentors at the internship site; 
and (4) alumni services consisting of individual coaching, alumni reunions and events, and a paid 
internship opportunity during the summer break from college. 

Characteristics of Program Applicants 

• Demographic characteristics: The average age for all applicants at the start of the program was 17. 
Ninety percent of applicants were black, and 65 percent of applicants were female. Over half lived 
only with their mother, 5 percent only with their father, one quarter lived in two- parent homes, and 
12 percent lived with neither parent. Four percent have children of their own. 

• Work experience: Three- quarters of youth reported at least some work experience before applying for 
the program, with average experience of just less than 10 months in all jobs combined. Common job 
experience was through summer jobs, including those accessed through the Summer Youth 
Employment Program in Washington, DC. 

• Educational background: Slightly more than a quarter of Urban Alliance applicants attended a 
charter school, and the majority of these were in Washington, DC. Applicants on average exhibited 
satisfactory but not stellar performance in school. The average cumulative GPA at the end of 
applicants' junior year was 2.66. Over one- third of applicants had attended more than one high 
school. 

• Motivations: Many program staff and youth participants cited the internship wages as a major 
motivating factor in applying to the program. Youth also reported interest in building up credentials 
on their resume for college or job applications. Many saw the internship and training as an 
opportunity to develop professional and interpersonal skills for future use, and some noted prestige 
among their social networks for having a professional internship. Generally, youth articulated they 
planned to go to college and choose professional career fields. 


Program Implementation 


• Recruitment and application: Collaboration between Urban Alliance and local high schools is 
important for recruiting participants. In Baltimore, a formal partnership with the schools allows 
youth to receive course credit for participation; in DC, the partnership is informal. In both sites, 
school counselors and other staff encourage the participation of "middle-of-the-road" students: those 
who are not at the top of the class but are not struggling severely. Youth must obtain an early- release 
schedule to participate, so they tend to have some flexibility with their schedules; their schedules are 
not filled with AP courses or unmet graduation requirements. 

• Pre- work training and workshops: Training is delivered in many formats, including lectures, group 
work, skits, and hands-on individual or group activities. 

o Pre- work training: This phase of training lasts four to six weeks and focuses on building job 
skills, including soft skills such as howto behave in an office setting and presenting a 
"professional self." 

o Training workshops during internship phase: Workshops focus on general life skills, self- 
sufficiency, and college planning. They keep youth motivated about going to college. 

o Public speaking challenge: At the end of the program, youth give presentations on their 
internship experience at the public speaking challenge and receive a bonus for successful 
completion. This provides them with an opportunity to showcase the skills they have gained 
in communication and public speaking. 

• Internship: Youth who complete pre- work training enter the internship phase of the program. Youth 
are matched with job sites based on the degree of nurturance they need, their distance to the job site, 
career interests, and personality. Typical job duties included filing, correspondence, shipments, 
copying, printing, and other clerical duties. These basic office tasks are considered part of the skills 
growth process. The internship is meant to provide a foundation of skills that can be used in future 
jobs in any field and to motivate youth to pursue higher education. 

o J ob mentors: Each job site selects a mentor to guide the youth through the internship. Some 
job mentors contribute to youth development in a wide-ranging mentorship role, some 
principally act as supervisors, and others provide general mentoring but little job- specific 
guidance. Some mentors reported speaking with their intern every day, but for others contact 
was weekly or monthly if other staff at the organization assigned the intern work. 
Communication between job mentors and Program Coordinators occurs throughout the 
program and is important to program success. 

o Attracting and retaining job sites: Urban Alliance Program Coordinators, senior- level staff, 
and board members all help attract organizations to serve as job sites for interns. Attracting 
sites is challenging and was especially difficult in Baltimore, but once involved, organizations 
tend to stay involved in subsequent years. 

o Intern value to employers: J ob sites found interns to be helpful additions to their workplace. 
Some reported that interns helped them meet their aim of increasing workplace diversity, 
boosted public image, or provided employees an opportunity to gain supervisory experience. 

o Internship value to youth: The potential value of the Urban Alliance internship for 

participating youth is substantial. The internship provides paid work: youth earned nearly 
$3,800 (in 2013 dollars) on average, participated in their internships for an average of 35 
weeks, and averaged about 400 hours of work. In addition, the internship provides access to 
an otherwise inaccessible, often prestigious, job environment. Some youth secure future 
internships or jobs as a result of their internship, and some gain connections at colleges. 
Youth also benefit from mentor support through tangible help with resumes; career advice; 
college applications; or encouragement to complete the program, school, or navigate 
hardships at home. Youth gain hard and soft skills, with the gains in soft skills more notable. 
The improvements most commonly mentioned by youth and by staff were in communication 
skills, confidence, and comportment. 
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• Program Coordinators: Program Coordinators manage a caseload of 30- 35 interns. Their duties 
include running training sessions; communicating with their caseload; entering tracking notes and 
other data into Urban Alliance's case- management system; connecting youth with resources as 
needed; communicating with j ob mentors and school counselors; and leading one- on- one post- high 
school planning sessions. Program Coordinators reported this set of responsibilities is both highly 
rewarding and highly demanding; many would prefer a smaller caseload and felt that the position 
induced burnout. Furthermore, many acknowledged difficulty in balancing time between encouraging 
youths' professional development and supporting their academic success. 

o Post- high school planning: Three times a year, youth meet with Program Coordinators for 
one-on-one 30- 60 minute post- high school planning sessions. During the sessions, they help 
youth with college essays and applications, applications for financial aid and scholarships, 
college choices, and preparation for the public speaking challenge. Many Program 
Coordinators felt there was insufficient time for post- high school planning. 

o Coaching relationship: The relationship between Program Coordinators and youth is 

extremely important and is believed to influence retention, commitment to college, and level 
of internship attendance. Program Coordinators strive to maintain a close, open, encouraging 
relationship with youth. Contact between youth and Program Coordinators varied in form 
and frequency, with high- need youth sometimes occupying the bulk of time staff could spend 
providing one- on- one guidance. Program Coordinators reported connecting with youth one to 
five times per week outside a weekly e-mail to their caseload. Program Coordinators believe 
most youth need very frequent communication as part of the coaching relationship. Though 
students expressed annoyance at the high level of contact, they also seemed to be grateful for 
its ability to keep them on track. 

o Communication with schools: Urban Alliance assigns a specific program staff member as the 
main point of contact with each school. Program staff contact the schools to provide updates 
and compare notes on youth. The frequency of communication with schools varies depending 
on the style of the school counselor and the challenges faced by particular Urban Alliance 
interns. 

o Qualifications, training, and retention: Program Coordinators have at least a bachelor's 
degree and usually have at least two years of direct service experience. They receive a week of 
initial orientation and a yearly stipend for further training. Most learning is on-the-job; 
Program Coordinators say that through a process of trial and error, they learned howto deal 
with different situations and types of students. Typically, Program Coordinators stay in the 
position for two to three years, though some have advanced to management level within the 
organization. 

• Program Attrition: Substantial attrition occurs in the Urban Alliance internship program. Of youth 
who completed an application and were admitted to the treatment group, more than one in five did 
not showup to pre-work training. Roughly one- quarter began but did not complete pre-work training. 
Of the 52 percent of accepted applicants who completed pre- work training, nearly all were placed in 
an internship. Overall, about two of five accepted applicants ultimately completed the program. The 
high rates of attrition before and during pre- work training are not seen as problematic by the 
program; youth with low motivation or irreconcilable scheduling conflicts leave the program before 
beginning an internship. The driver of attrition most commonly mentioned was the cost of 
transportation to and from training events and job sites. Scheduling demands were also a problem, 
especially in the pre- work training phase. Further, some youth did not complete their jobs because of 
poor attendance; misconduct; or conflicting employment, educational, or personal obligations. 

• Program Costs: Youth participant wages, awards, and fringe costs together made up nearly half of all 
program costs. A few job sites pay youth wages themselves, though for most. Urban Alliance covers 
the cost. Staff wages and fringe benefits totaled another 42 percent of costs; the remainder was rent, 
administrative costs, and other direct program costs. Cost per student is $4,925 if spread across all 
youth who attended at least some pre- work training, but equals $8,866 when spread only among 
youth who completed the entire program. 
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• Organizational growth and change: Urban Alliance has experienced tremendous growth over the 
past decade, transforming itself from a local nonprofit with four staff members to a multisite 
organization with 42 staff members that has served over 1,500 students. Urban Alliance enters new 
sites if staff believe there is a need in that locality, at least 70 internships can be secured, and the 
regional staffing can be put in place. The high school internship program now operates in 
Washington, DC; Baltimore; Chicago; and Northern Virginia. Staff members say it can be challenging 
to navigate the program model in each site; a newly created chief program officer position helps 
ensure that the sites implement the curriculum and share information in a standardized manner. 
Growth within a particular site is dependent on the number of local internship slots that can be 
secured for students. Besides expanding the number of students at each site. Urban Alliance has 
experienced growth in the technologies and tools that each site uses to serve its youth. In 2012, for 
instance, sites began using Salesforce's web- based case- management system. The organization has 
also developed new capacities in evaluation and alumni services. 

Implications 

• Urban Alliance fills in important niche in the continuum of services provided to at- risk youth in two 
ways. First, it serves "middle-of-the-road" students who are likely to graduate, but who may have 
difficulties acguiring good jobs or enrolling in higher education after high school. Second, Urban 
Alliance has created an intermediary role between schools and employers, relieving schools of a task 
for which they may be ill-suited. Employers are able to deal directly with a responsive organization 
that will provide interns with a beginning set of both hard and soft skills. 

• The internship program depends heavily on buy-in from schools and employers. School 
administrators must buy into the model in order to allow for the necessary early- release schedules, 
and school counselors must help identify appropriate program applicants. Employers must be willing 
to welcome low- skilled high school students, give them genuine work opportunities in an office 
setting, and negotiate their issues with the help of Urban Alliance. 

• The office- based work experience Urban Alliance provides helps youth understand how college can 
improve their future opportunities and earnings. However, if college is the goal, more could be done 
to introduce students to college, such as sponsoring campus visits. 

• The Urban Alliance funding scheme is an important component of the Urban Alliance model. By 
having employers pay Urban Alliance a contribution. Urban Alliance can oversee the payment of 
wages to youth and cover the additional costs of administering their high school internship program, 
allowing the organization to achieve sustainability of the program after initial start-up. 

• Program attrition is high and poses challenges. For one, attrition rates vary each year, so Urban 
Alliance may end up with either too few or too many youth for the number of internships they have 
available. Perhaps more important is the possibility that the program serves a motivated group of 
youth who may have done well in the absence of the program. Although these youth can still benefit 
from the training and internships, the resources may be better allocated to serve more at-risk youth. 
The results of the impact evaluation will help determine the appropriateness of the self- selection 
aspect of Urban Alliance's program design. 


4 


URBAN INSTITUTE 


Introduction 

Effective programs to help disadvantaged youth become self-s uffi cient are critical. Urban Alliance, 
headguartered in Washington, DC, serves such youth through its high school internship program, which 
provides high school seniors from distressed communities with training, mentoring, and work experience 
to help them successfully transition to higher education or employment after graduation. Youth growing 
up in low- income and low- opportunity communities, such as those targeted by Urban Alliance, face 
formidable challenges in graduating high school and transitioning to adulthood despite some recent 
efforts to revitalize disinvested neighborhoods and reform struggling school systems. In Washington, DC, 
where the cost of living is high, over a guarter (26.5 percent) of children under age 18 live below the 
federal poverty level. In Baltimore, the share of children in poverty is even higher, at 34. 1 percent. 1 The 
majority of students in both cities attending public and public charter schools receive free or reduced- 
price lunches: 72 percent in DC and 85 percent in Baltimore in 2012. 2 

Many of the schools in cities where Urban Alliance operates have poor academic outcomes. In both 
Baltimore and DC public schools, around two- thirds (69 percent and 58 percent, respectively) 3 of those 
who enter ninth grade graduate within four years. The students who make it to graduation are often 
unprepared for life after high school. Many high school seniors in both cities' public school systems are 
not proficient in such core subjects as math and English. Unsurprisingly, many of DC and Baltimore's 
young residents do not go on to college, face unemployment, and have limited options for future skill 
development. 

Since its founding in 1996, Urban Alliance has placed over 1, 500 youth in internships, growing to 
serve over 350 interns annually in four sites: Baltimore (since 2008), Chicago (2012), Northern Virginia 
(2013), and its original site, Washington, DC. As part of this expansion process. Urban Alliance has 
commissioned the Urban Institute to conduct a six- year, randomized controlled trial impact and process 
evaluation of its high school internship program. This report provides baseline information about Urban 
Alliance and the youth participating in its high school internship program in Washington, DC, and 
Baltimore in two consecutive program years. ( Since this research began, the organization opened program 
sites in Chicago and Northern Virginia, but these are not included in this study.) This baseline report also 
presents a process analysis of the program, informed by extensive evaluator observation and interviews 
with staff, stakeholders, and youth. This analysis describes in detail the program model, which has 
remained essentially the same throughout the organization's significant expansion, and analyzes the 
program's implementation. It also serves as the starting point of a longer impact study that will determine 
and describe the early- adulthood impacts of the Urban Alliance internship program on the youth it serves. 

Background 

A sizable literature exists describing the attributes and impacts of different programs designed to improve 
the educational and career outcomes of youth, many of which involve work- based learning. Programs 
focus on many types of youth, including high school students, dropouts, and youth who have experience 
in the juvenile justice or child welfare systems. Some of these programs, like Urban Alliance, are run by 
private organizations or social service departments, rather than through collaborations with the education 
system. Other programs, such as career academies and magnet schools, are coordinated within high 
schools and offer workplace skills and experience as a part of secondary education. Regardless of the 
program setting, the features of the programs and the characteristics of the student participants vary 
widely. 

Although no studies have rigorously evaluated a program with Urban Alliance's unigue combination 
of work experience, training, mentorship and coaching, and alumni support, studies have assessed 
programs offering different combinations of these supports. Several studies have not yielded evidence of 
positive long-term outcomes. However, many studies have only tracked outcomes in the short term, and 
the major federal evaluations of youth employment programs have focused on programs geared toward 
disconnected youth rather than youth still in traditional high school settings. There is still much to be 
learned about programs such as Urban Alliance, which provide a comprehensive and intensive array of 
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services to students who are still in high school and have not yet become disconnected from education or 
employment. 

Starting with work experience, a review of research on the effect of employment on youth academic 
and career outcomes, outside of any structured program, shows that there may be a positive relationship 
between employment during high school and later outcomes. Some longitudinal studies have shown that 
holding a job during high school is associated with higher academic success. Light ( 1999) finds that 
students with jobs during high school who worked a moderate number of hours per week (less than 20) 
perform better in school than students who do not work at all. Ruhm ( 1995) finds that students working 
20 hours per week have significantly higher earnings six to nine years later. Rothstein (2001) finds 
positive relationships between teenage employment and future employment and education: teens with a 
moderate level of work at age 16- 17 worked about six more weeks per year at age 18- 30 than those who 
had not worked as teens. Furthermore, teens who worked up to 20 hours per week were more likely to 
have at least some college education by age 30. However, later studies find no positive correlation 
between teen and later adult employment (Rothstein 2007; Tyler 2003). 

Secondary education programs that connect students to internships, combine learning with a job, or 
in some other way provide youth with an on-the-job learning experience can prove beneficial. Career 
academies— partially self-contained occupationally themed subschools within high schools— generally are 
associated with better attendance and grades. One study of career academies found attendees are 22 
percent more likely to exhibit positive school performance than students at comprehensive high schools 
(Crain et al. 1998). In a randomized controlled trial of career academies, Kemple (2008) found that 
participants experienced higher levels of interpersonal support from peers and teachers, and those 
students who entered school at high risk of dropping out were more likely to stay through 12th grade. 
Eight years after entering the program, participants had earnings and employment higher than non- 
academy students in their high schools. A study of school administrative data found that students in 
career academies were 9 percent more likely to graduate and 18 percent more likely to attend a 
postsecondary institution (Maxwell and Rubin 1997). Similarly, studies of career magnet schools, which 
specialize in one particular career theme (such as agricultural science or business), have found that they 
result in lower dropout rates and increased student investment in school (Katz et al. 1995). Findings have 
been mixed on whether they improve academic achievement (Ballou, Goldring, and Liu 2006; Cobb, 
Bifulco, and Bell 2009). In comparing the quality of school- based employment with outside employment, 
one study finds that school- based jobs are lower in quality but do offer important work experiences to 
youth who would have difficulty finding work on their own (Hamilton and Sumner 2012). 

Some work- based learning programs operating outside schools have been shown to increase the 
academic performance and classroom attendance of participating students while decreasing delinquent 
behaviors outside class. One study found a positive effect of local government internships on test scores 
(Hamilton and Zeldin 1987). A randomized controlled trial evaluation of New York City's Center for 
Economic Opportunity youth literacy program found that students with a paid summer internship to 
complement their literacy, math, and job skills education attended more class hours and improved their 
math grade a full letter grade more than those without the internship (NYC Center for Economic 
Opportunity 20 11) . A summer youth employment program in Boston was found to reduce adverse social 
behaviors such as violence and druguse (Sum, Trubskyy, and McHugh 2013). A random assignment 
evaluation of Youth Corps, a federally funded program providing paid jobs for youths age 18- 24, with 
academic support for those needing a GED (General Educational Development) certification, 4 found no 
impacts on educational attainment or employment in an 18- month follow-up survey. However, 
participants were 7 percentage points more likely to report that they planned to complete at least some 
college (Price et al. 2011). 

There are few evaluations of programs combining an internship with other academic or social 
supports. An evaluation of After School Matters, which offers high school students paid apprenticeship- 
type experiences in many settings, found no impacts on marketable job skills or academic outcomes — 
though it did find a reduction in problem behaviors and more markers of positive youth development 
amongthe treatment group (Hirsch et al. 2011). The Summer Career Exploration Program in 
Philadelphia, which provides high school students with a summer job in the private sector, pre- 
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employment training, and a college student mentor, found no impacts on students' high school 
graduation, college enrollment, attitudes toward work or school, or sense of self-efficacy. The program's 
only positive impact was that participants were more likely to enroll in a college preparatory or specialized 
academic program ( 12 percent, compared with 8 percent for the control group; see McClanahan, Sipe, and 
Smith 2004). It is unclear whether programs like the Summer Career Exploration Program, which lasted 
only for the summer, would have been more effective if they were longer-term. A quasi- experimental 
study of a Boston school- to- career initiative offering youths intensive academic instruction, worksite 
learning experiences, and post- high school supports found positive impacts for youths who participated 
in the program, compared with a control group of youths who would have met the program's eligibility 
standards had they applied. It found that the program group members were 6 percentage points more 
likely to attend college, with an even more pronounced positive effect for African Americans (J obs for the 
Future 1998). 

Programs that offer youthjob training without direct job experience have documented some success. 
Participation in J ob Corps, a federally funded program providing vocational training, academic support, 
counseling, and often residential living, was found to have short-term impacts on earnings, employment, 
education, and crime. However, after 5 to 10 years these impacts disappeared for the sample as a whole, 
containing youth ages 16- 24 at the time of application, with the impact on earnings remaining significant 
only for the subgroup ofyouth ages 20- 24 (Schochet, Burghardt, and McConnell 2006). Additionally, an 
impact study of the J ob Training Partnership Act, a previous federal program, found positive impacts on 
adult earnings and employment but little or no effect on youth employment or the earnings of female 
youth; it also found a negative impact on the earnings of male youth (Bloom et al. 1993). An evaluation of 
replications of San J ose's Center for Employment Training (CET), a training program for high school 
dropouts, found no lasting impact of the program on earnings or employment (Miller et al. 2005)— though 
the authors note this may have been caused by widespread infidelity to the CET program model. Two 
studies of the CET program in the early 1990s did find positive impacts on employment and earnings 
(Burghardt et al. 1992; Cave et al. 1993) . Perhaps the most promising recent job training program 
evaluation is that of the National Guard Youth ChalleNGe program. This program, which provides short- 
term job and life skills training in a quasi- military environment and includes follow-up mentoring, 
demonstrated long-term positive effects on employment specifically: after three years, the program group 
had an employment rate 7 percentage points higher and earnings 20 percent higher than a comparison 
group, and program participants were more likely to obtain college credits or a high school diploma or 
GED (Millenky et al. 2011). 

Another common approach to serving at-risk youth is to provide case management and mentoring, 
and such programs have documented generally positive results at least in the near and medium term. 
Impact studies of the Big Brothers Big Sisters mentoring program have reported mixed findings. One 
study found that treatment group members skipped half as many days of school, had slightly better GPAs, 
and had an improved concept of their scholastic competence (Tierney, Grossman, and Resch 1995). 
Another study also found improved academic confidence and performance, but only at first; impacts 
disappeared by 15 months (Herrera etal. 2011). Grossman and Rhodes (2002) found that youths enrolled 
in the program for more than 12 months had significant gains at 18 months in self-worth, perceived 
scholastic competence, relationships with parents, and other positive social outcomes. A specialized Big 
Brothers Big Sisters program for children of incarcerated parents found treatment group youths had 
higher self-esteem and felt more connected to school, community, and family at 18 months, but they did 
not differ in their academic competence or attitudes (US Department of J ustice 2011). A Philadelphia- area 
program that provides mentoring for all four years of high school found that students offered a mentor 
had college attendance rates in the first two years after high school graduation that were 20 percentage 
points higher than those of their peers (J ohnson 1999) . 

College access and readiness programs, which aim to help students graduate high school and enroll in 
college prepared for the challenge, have had mixed results as well. Upward Bound, a federally funded 
program lasting up to four years and offering instruction, tutoring, and counseling, was found to have no 
overall impact on high school graduation or college enrollment. However, it was found to improve 
education outcomes for students with initially low educational expectations. These students were more 
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than twice as likely to enroll at four-year colleges (38 versus 18 percent) than similar control group 
members (Myers et al. 2004). The random assignment evaluation of the Quantum Opportunities Project, 
which operated in five sites across the country and offered case management, academic support, 
developmental activities, and community service, found no positive impacts, though this was attributed to 
poor implementation of the program model and low participation (Schirm et al. 2006). Harvill and 
colleagues' (2012) meta-analysis of 14 college access program experimental or guasi- experimental 
evaluations found an average boost to high school graduation rates of 8 percentage points. However, 
when only the three experimental evaluations are considered, the impact is not statistically significant. 
The analysis also found an average increase in college enrollment of 12 percentage points, whether all 
evaluations or only the experimental ones are considered. 

In all, there is evidence to suggest that programs offering underserved youths jobs, job training, 
career- focused education, mentoring, or college readiness activities— or some combination of these— may 
be effective in helping youth achieve better outcomes. However, we know little about the effects of 
intensive initiatives for students still in high school that provide not only a paid job, but also 
individualized support and continual training. From the existing evidence, it is difficult to determine if the 
sort of outcomes achieved by the National Guard Youth ChalleNGe program— that is, positive impacts on 
actual earnings and employment— could be achieved by a program that takes place during the school year 
and does not include a residential component. A rigorous evaluation of the Urban Alliance internship 
program will help us know more. 

Urban Alliance High School Internship Program Model 

Urban Alliance has developed a program model designed to address the organization's goals of 
empowering underresourced youths to aspire, work, and succeed through paid internships, formal 
training, and mentoring. Urban Alliance targets its internship program to a subset of high school- age 
youth, reaches them through a months- long recruitment process, and delivers four main program 
components. 

Target Population and Recruitment 

Urban Alliance targets students at selected schools (further described in the "Urban Alliance Applicants" 
section on page 8) that they consider to have a high proportion of at-risk youth. The organization seeks 
out youths who will be in their senior year of high school during the upcoming program year. These 
youths will need to have enough course credits accumulated to allow for an early- release schedule during 
the internship phase of the program. The Urban Alliance program is targeted to high school seniors 
because program staff believe the program is most effective at reaching young people during this 
transitional year; its lessons and curriculum are designed for youths about to enter adulthood. In 
addition, the program targets youths in their senior year because only by that point will they have 
accumulated enough credits to have a shortened school day schedule. 

The program aims to serve "middle-of-the-road" high school students who maintain a GPA of 2.5 or 
above, but it is not limited to that group. Although Urban Alliance leadership believes that students with 
GPAs that are too low will generally have insufficient time, resources, and course credits to participate in 
the program and graduate on time, the program often accepts youths with lower GPAs. The program also 
does not exclude youths with high GPAs, though these students often cannot participate in the program 
because, although they may have sufficient credits to move to an early- release schedule, they are more 
likely to be taking honors and Advanced Placement courses to fill up their schedules. Youths with high 
grades may also have higher skill levels and more external support, so their need for the program may be 
lower. 

Urban Alliance begins to recruit students for its high school internship program in the spring of 
students' junior year, and recruitment continues into the fall of their senior year. The recruitment process 
differs between cities. In Washington, DC, the organization's relationship with schools is informal. Urban 
Alliance presents its program during assemblies or in classrooms to high school juniors at several public 
and charter schools in the city. In Baltimore, the relationship with the school system is formalized, and 
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youths receive course credit for participating in the program. School counselors and teachers identify and 
refer students in their schools who they think will benefit most from the program. Many of these youth do 
not formally apply until they start pre- work training. In the more recently opened Chicago program. 

Urban Alliance has also established a formal partnership with the city's school district, and youths receive 
course credit for participating in the program. 

Program Components 

The Urban Alliance high school internship program has four primary components: training, paid 
internships, coaching and mentoring, and alumni services. 

Training: Pre-work and Workshop 

Urban Alliance conducts training workshops from late September or early October of each school year 
through the end of J uly. This training includes three to six weeks (varying by city) of "pre- work" training 
before the start of the internship. Program participants are expected to attend training for one to one and 
a half hours every day after school during that period. The primary goal of pre- work training is to prepare 
the youths for their internships. Topics include workplace etiquette and culture, as well as such hard skills 
as faxing and Microsoft Excel basics. Urban Alliance also uses these sessions to familiarize youth with 
post- high school education and employment options, financial literacy, and select life skills. Sometimes, 
youth are assigned homework. During pre- work training, youth receive training on j ob interviewing, 
which they then use in interviews with mentors at their prospective job sites. 

After the internships start in the late fall, youth are expected to attend workshops every Friday after 
school. Workshops focus heavily on topics related to post- high school planning, financial self-sufficiency, 
and life skills, though they also continue to review workplace- relevant topics. After the school year ends, 
youth attend half-day workshops every Friday. 

Urban Alliance staff also prepare youth for a final presentation that interns give in July at Urban 
Alliance's public speaking challenge event. These are PowerPoint presentations designed by the youth to 
describe their recent internship experiences and career goals. A volunteer panel of community 
stakeholders judges the youth, who can receive a $ 100 prize for performance. Youth can also receive 
bonuses earlier in the year for participating at other events, referring friends to Urban Alliance, or 
participating in program activities while waiting on a delayed job placement. 

Paid Internships 

Urban Alliance program staff pair students who complete pre- work training with paid internships based 
on each student's skill levels, needs, interest, and the range of internships available. Starting in the late 
fall. Urban Alliance participants go to their internships from 2:00 to 5:00 p.m. after school Monday 
through Thursday. This schedule requires that interns obtain permission for an early- release class 
schedule during their senior year of high school. During the summer following graduation. Urban Alliance 
interns work full days Monday through Thursday. Urban Alliance partners with professional clothing 
nonprofits such as Dress for Success to give interns access to clothing. 

The settings and responsibilities for internships vary, but most are office settings and include such 
tasks as filing, copying, and answering phones. Other types of jobs include greeting and directing guests in 
hotels or banks. Some interns also work in educational or day care settings. Interns earn a starting hourly 
wage close to their city's minimum wage ($8.25 in DC and $7.25 in Baltimore), which can rise to $10.00 
based on job performance and effort toward Urban Alliance activities, including workshop attendance and 
communication with their assigned Program Coordinator. For the most part, interns are officially 
employed and paid by Urban Alliance while working at their internship sites, though select job sites pay 
interns directly. 

Coaching and Mentorship 

Youth receive job mentoring and general coaching as part of the program. In addition to running the 
training workshops described above, front-line staff (Program Coordinators) maintain coaching 
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relationships with each youth assigned to their workshop group. Each Program Coordinator has a 
caseload of approximately 30- 35 interns whom they support throughout the program. Coordinators track 
individual student performance in a number of areas including workshop and job attendance, punctuality, 
workshop homework assignments, academic progress, post- high school planning, and progress toward 
the presentation for the public speaking challenge. Program Coordinators also send out a weekly e-mail to 
youth, and youth must check in with Program Coordinators at least once during the week. If interns are 
going to be late to work or miss work, they must contact their Program Coordinators and their employers. 

The Program Coordinators meet with each intern three times per year in a one- on- one meeting to 
discuss post- high school planning. They also provide ad hoc support, speaking with youths before or after 
workshops sessions; discussing youths' experiences in a group during workshop; or keeping in touch via 
individual phone calls, e-mails, and texts. Some youth face serious challenges such as teen pregnancy, 
domestic or relationship abuse, problems with their home life, or housing instability. Program 
Coordinators support youth emotionally and connect them with external resources to meet their needs. 

Each intern is also assigned to a "job mentor" or supervisor, who is an employee at the intern's 
workplace responsible for ensuring that the intern has adequate and appropriate work, teaching the 
intern necessary skills, and, ideally, providing opportunities for enrichment and networking within the 
workplace. J ob mentors assess interns' performance in the workplace. J ob mentors may suggest possible 
termination of an intern's position if his or her attendance or performance is poor, but the program 
endeavors to resolve all performance issues except the most severe (e.g., time- sheet fraud). When 
performance concerns arise. Urban Alliance staff first establish a work contract with the youth. Only if 
poor performance persists after several intervention attempts will the organization fire the youth and ask 
the intern to leave the program. 

Alumni Services 

As the Urban Alliance program has grown and more youth have completed the program and transitioned 
to life after high school. Urban Alliance Program Coordinators have increasingly found themselves 
providing informal support to youth who keep in touch after graduating from the program. In 2007, 

Urban Alliance first began offering informal education and career support services to alumni. More 
recently, it form aliz ed this program component, adding a full-time alumni services manager and city- 
specific half-time coordinators in 20 11. Through alumni services. Urban Alliance aims to prevent program 
alumni who are college students from dropping out and to link alumni with work. Alumni services also 
provide an avenue for tracking student outcomes after program completion. 

Urban Alliance services for alumni include ad hoc individual coaching meetings with youth, a 
resource room where alumni can access job search and education materials, networking opportunities 
through a website, alumni reunions, and connections to paid internship opportunities. 

Logic Model 

Initially, Urban Alliance measured its success sporadically and informally (Winkler, Gross, and Theodos 
2009), but the organization developed a formal logic model that details how it expects program activities 
to lead to specific outputs and, ultimately, outcomes for the youth served (figure 1) . The logic model 
describes both the four main activities that youth engage in and a set of outputs and related targets 
associated with them. For example, the first activity is to place students in professional, paid internships 
to expose them to the world of work, and one target is that 70 percent of students invited to pre-work 
training complete it and are placed at a jobsite. Other outputs relate to the development of work skills and 
initiation of post- high school planning. As the logic model demonstrates, the majority of expected short- 
and long-term outcomes relate to postsecondary education. Program staff articulate they hope most youth 
will first complete college before becoming employed; they also assert that the employment- readiness 
training is valuable even for those youths who elect to enter the labor force rather than attend college or a 
technical or training program. 

The next three columns in the logic model highlight the short-term, intermediate, and long-term 
outcomes and indicators associated with this activity. In its early years. Urban Alliance developed five 
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goals for participants in its high school internship program. Specifically, youth would ( 1) improve their 
hard and soft job skills, (2) gain long-term employment experience, (3) graduate from high school, (4) 
attend college or a training program, and (5) identify long-term employment opportunities. 


Figure I. Urban Alliance High School Internship Program Logic Model 


Activities/outputs 


Pre-work training 

• 90% of students completing 
pre-work receive 40 hours of 
training 

Professional internship 

■ 80% of students placed at a 
jobsite complete the program 

• 75% of interns work 500 hours 
or more annually 1 

• 75% of interns attend 90% of 
workshop trainings 1 

• 75% of interns receive four skill 
set evaluations from mentors 1 

• 75?4 of interns are exposed to 
at least 7 out of 1 0 hard skills 
on their final evaluation 6 

Post-high school coaching 

• 75% of interns 1 have a 
completed post-high school 
plan 6 

Alumni services' 

■ 90% of “disconnected youth” 
receive corrective action 
within six months after high 
school graduation 

• Alumni services department 
receives updates on 90% of 
graduating cohort through 
response to alumni survey, 
NSC record, or follow-up calls 


Short-term 

outcomes 


Skill set growth 

• 85% of youth demonstrate growth in 
both hard and soft skills between 
baseline and final evaluations or end 
with a 3.75 out of 4 in all 
assessments 6 

Post-high school planning 

• 95% of interns complete a college 
application 6 

• 95% of interns complete the FAFSA 6 

• 95% of interns apply to at least 
one scholarship 6 

• 95?4 of all interns have a final resume 
and cover letter at program 
completion 6 

Postsecondary outcomes 

• 1 00% of seniors completing the 
program graduate from high school 

• 85% of graduating cohort is 
“connected" by enrolling at least half 
time in postsecondary education or 
earning a wage of $ 1 5/hr. 


Intermediate-term 

outcomes 


Long-term outcomes 


80% of alumni are 
“connected" one year 
after the program 
70% of alumni are 
"connected” two years 
after the program 
I st-year college 
persistence rate: 70% 


70% of students who 
enter four-year 
college graduate 
within six years of 
entry 

70% of students who 
enter two-year 
colleges graduate 
within three years of 
entry 

80?4 of program 
participants are living 
self-sufficiently and 
earning a livable wage, 
independent of 
welfare, EITC, or food 
stamps 



Source: Urban Alliance. 

NSC = National Student Clearinghouse; FAFSA = Free Application for Federal Student Aid; EITC = earned income tax credit. 
a Out of all interns placed at job sites. 
b Out of all interns completing the program. 

c “Alumni services corrective action” is a one-on-one session with an alumni services representative that includes resume review, interview 
preparation, budget planning, and counseling. 

d Post-high school plans measure an intern’s readiness for college or employment. Finalized post-high school plans will have all the career or 
education preparation steps completed. 


A final note about the program design relates to its funding: the internship program is financed 
directly by internship sites and by philanthropic foundations. Approximately 75 percent of internship 
placement sites, most typically for-profit businesses but also nonprofit and governmental organizations, 
make a donation to Urban Alliance for each intern they hire; in DC the expected amount is $ 12,500 and in 
Baltimore it is $10,000. This donation is tax-deductible for the for-profit firms. Urban Alliance uses this 
contribution to pay intern wages and to cover the costs of services provided to the interns. It raises 
additional funds to cover the cost of placing students at job sites that cannot afford the $ 12,500 donation. 
Under the current model, sites must have approximately 70 internship slots to be sustainable financially. 

Data and Methods 

This baseline report is part of a larger six- year impact and process evaluation of the Urban Alliance high 
school internship program. This report reviews two program years, combined for analytic purposes: 
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20 11- 12 and 20 12- 13, where program years correspond roughly to school years. The decision to combine 
two program years was necessitated by insufficient sample size for impact measurement in any single- 
year cohort. The analyses focus solely on the Baltimore and Washington, DC, sites, because the 
organization's expansion to Chicago in 2012 and Northern Virginia in 2013 happened after the evaluation 
had begun. 

For the purpose of this study, researchers assigned 20 11- 12 and 20 12- 13 program applicants in 
Washington, DC, and Baltimore at random to a treatment group or a control group. Random assignment 
was possible because demand among youth exceeded the number of participants the program could 
support. Those assigned to the treatment group were invited to participate in the program, which begins 
with mandatory pre-work training before assignment to an internship position. Youths in the control 
group were not invited to participate in the training or internship. In describing baseline characteristics of 
youths in this report, we include youths in the treatment and in the control group in our sample— a total of 
1,062 youths— while the process analysis is based on conversations with only youth in the treatment 
group. 

Researchers collected data from a number of sources, using a combination of guantitative and 
gualitative data collection methods, further detailed in appendix A. Program applications, collected from 
all applicants, included baseline demographic and personal characteristics such as race and ethnicity, 
household structure, home address, and career interests. The addresses were matched to census tracts 
and accompanying indicators from the American Community Survey on unemployment, poverty, and 
racial and ethnic composition to determine characteristics of youths' neighborhoods. Service delivery data 
collected internally by Urban Alliance tracked case management status and participation in program 
activities, noting youth attendance, progress in completing post- high school planning actions such as 
submitting applications for financial aid, and other important indicators. Financial records of biweekly 
wages paid to interns were also collected and were used to more accurately determine youths' status in the 
program during the internship phase. While program applications included items assessing school 
attendance and GPA, school system student- level data were also collected to verify these figures and to 
add additional indicators such as whether students were in a special education program. 

Nearly 50 in-depth interviews and focus groups with Urban Alliance students, staff, job mentors, and 
school counselors covered many topics regarding the program model, implementation, and challenges 
facing participants. To see components of the program in action first hand, we also observed 12 pre-work 
training and workshop training sessions, noting student engagement, teaching technigues, staff 
preparedness, and material covered. From these sessions and from other interactions with program staff, 
we gathered Urban Alliance printed materials, including logic models, materials for training workshops, 
and alumni newsletters were also reviewed. Finally, we collected official audited financial records for the 
2012 fiscal year to better understand the allocation of costs associated with the Urban Alliance internship 
program. 

Impact Study: Looking Ahead 

This report is the first of three reports evaluating the implementation and impacts of the Urban Alliance 
high school internship program. Key outcomes assessed by the impact study will include high school 
graduation, college enrollment, college persistence, employment, and healthy behaviors. (We will not be 
able to measure attainment of a four- year college degree and post- college employment as these outcomes 
will occur outside the study window.) 

To ascertain life experiences of Urban Alliance applicants during and after high school, we will use 
individual- level records from youths' high schools (including graduation and GPA) and National Student 
Clearinghouse data to track students' college enrollment and persistence. The Urban Institute is also 
conducting an extensive survey of study participants approximately 10 months and 30 months after their 
predicted high school graduation dates. We will participant outcomes using both "intent to treat" and 
"treatment on the treated" analyses. Interim results will be reported in 2015, with final results in 2016. 
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Urban Alliance Applicants 

This section describes the characteristics and motivations for participation among the 1,062 youth who 
applied to the Urban Alliance Program and participated in the evaluation. Because there were not 
significant differences in youth attributes by program year or site, applicants from the 2011- 12 and 2012- 
13 cohorts, from both Baltimore and DC, are pooled. However, characteristics of youths' neighborhoods 
and schools are presented by site, given some significant differences between the two. 

Characteristics 

The average age for all applicants at the beginning of pre-work training was 17 (table 1). The vast majority 
of applicants were black (90 percent), with small shares of "other" race/ ethnicity (5 percent), Hispanic (4 
percent) and white, non- Hispanic ( 1 percent) . Nearly all applicants were US citizens and less than 10 
percent reported speaking a second language at home other than English. 

Table I. Demographic Characteristics of Urban Alliance Applicants (percent, 
except where noted) 


Percentage 

Female 

64.7 

Age (average, years) 

17.6 

Race 


Black 

89.0 

Hispanic 

5.6 

Other 

4.0 

White 

1.5 

Speaks language other than English at home 

7.7 

US citizen 

99.3 

Has a bank account 

38.0 

Has previously held a job 

75.3 

Months worked at previous jobs (average, if held) 

9.7 

Has a child 

4.2 

Living arrangement 


Mom only 

56.2 

Two parents 

26.6 

Dad only 

5.2 

Other relative/guardian 

12.0 

Number of household members (average) 

4.3 

Employed adult in household 

77.4 

Observations 

1,062 


Source: Completed Urban Alliance high school internship program application forms for program years 201 1-12 and 2012-13. In 
cases where applications were missing data on age and race, data provided by DC Public Schools, Baltimore Public School Board, 
and DC Public Charter School Board were used. 

Notes: Estimates include applicants assigned to the treatment group and the control group. All items had a response rate of 90 
percent or more except bank account (71 percent) and employed adult in household (84 percent). Variance is due to nonresponse 
for some items on the application form. 


About two- thirds of Urban Alliance applicants were female. Given that the recruitment process is not 
aimed at either female or male students in particular, the factors underlying this discrepancy are unclear. 
However, when looking into other programs geared toward high school students, we see that they also 
typically serve more female than male youth (e.g.. After School Matters [59 percent female; Hirsch et al. 
2011], the Summer Career Exploration Program [62 percent female; McClanahan 2004], and Upward 
Bound [72 percent female; Schirm, Stuart, and McKie 2006]). 5 

Urban Alliance applicants largely did not live in two- parent homes ( 73 percent) . The maj ority lived 
with their mother as sole guardian ( 56 percent), and 5 percent with their father as sole guardian. Twelve 
percent lived with a grandparent or guardian other than a parent. Some Urban Alliance applicants came 
from stable homes, but others did not. One Program Coordinator, discussing the family- related challenges 
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that Urban Alliance youth face, characterized some students' home environments as so toxic that she 
wondered "if some of [these] young people should be emancipated." Other staff members recounted 
needing to support students who faced unstable housing due to families losing their homes to foreclosure 
or eviction. Multiple staff members acknowledged the many barriers and responsibilities these youth face. 
"Usually they're taking on more than just the role of a child," said one DC Program Coordinator. "They're 
taking maybe the role of a parent, in a way, or helping with younger siblings... [they have] more 
responsibilities than other young people." Nearly 4 percent of applicants are themselves parents, and 
others have caretaking responsibilities for siblings. 

Urban Alliance applicants typically come from underemployed households, with nearly a quarter of 
students reporting that no adults in their household were employed. Still, about three-quarters of 
applicants reported at least some prior work experience of their own, with average experience of just less 
than 10 months in all jobs combined. Most typically these positions were summer jobs, including jobs 
accessed through the Summer Youth Employment Program in Washington, DC. About four in ten youth 
reported having a checking or savings account, and a greater portion of youth with job experience (42 
percent) than of youth with no job experience (30 percent) reported having an account. 

Applicants typically resided in economically distressed neighborhoods (table 2). More than three- 
quarters (77 percent) lived in a neighborhood with an unemployment rate greater than 10 percent, and 
nearly half lived in neighborhoods with poverty rates higher than 25 percent. Maps of program applicants' 
homes in Baltimore and Washington, DC (figure 2), reveal that almost all reside in census tracts that are 
over 75 percent minority (dark green shading). Most applicants resided in communities with higher- than- 
average poverty, though a sizable portion of applicants lived in moderate- poverty areas (medium- light 
blue shading). Almost no youths hailed from low- poverty sections of Baltimore or DC. As one program 
administrator described, "some of them are living in moderate- income, mostly African American 
communities, [and] some of them are living in the toughest communities in the District." Generally, staff 
members believe that youths' upbringings in largely segregated and low- income neighborhoods limit their 
opportunities for socioeconomic mobility. As one senior member explained, most participants "haven't 
left their neighborhood," in the sense that they have had very little exposure to opportunities found in 
middle- and upper-class communities. 


Table 2. Characteristics of Urban Alliance Applicants’ Neighborhoods 



Baltimore 

DC 

Both sites 

Minority share 

Less than 25% 

3.5 

1.7 

2.0 

25%-50% 

6.5 

4.1 

4.6 

50%-75% 

1 1.5 

8.9 

9.4 

More than 75% 

78.5 

85.3 

84.0 

Share in poverty 

Less than 1 0% 

8.5 

1 1.8 

1 1.2 

1 0%-25% 

41.5 

39.2 

39.7 

25%-40% 

39.5 

33.1 

34.3 

More than 40% 

10.5 

15.8 

14.8 

Share unemployed 

Less than 5% 

3.0 

3.2 

3.2 

5%- 1 0% 

13.0 

21.4 

19.8 

1 0%-20% 

54.5 

37.9 

41.1 

More than 20% 

29.5 

37.5 

36.0 

Observations 

201 

847 

1,046 


Sources: Neighborhood characteristics are five-year averages at the tract level from the American Community Survey, 
2008-12, US Census Bureau. Tract determinations are based on youth addresses as reported on Urban Alliance high 
school internship program application forms. 

Notes: Estimates include applicants assigned to the treatment group and the control group. Sixteen applicants were omitted 
owing to incomplete address information. 
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Figure 2. Neighborhood Characteristics of Urban Alliance Applicants 
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Sources: Percent minority and percent poor are five-year averages at the tract level from the American Community Survey, 2008-12. Tract 
determinations are based on youth addresses as reported on Urban Alliance high school internship program application forms. 

Note: Dots represent the number of program applicants within each census tract and are placed randomly within each tract to display the 
relative distribution of applicants across region. 


Slightly more than a quarter of Urban Alliance applicants attended a charter school, and the vast 
majority of these were in Washington, DC, where about one third (33.7 percent) of applicants attended 
charter schools. Applicants on average exhibited satisfactory but not stellar performance in school, as 
shown in table 3. The average cumulative GPA at the end of junior year was 2.66, according to school 
records if available, or as reported on the application by a school counselor or by the student herself. 6 A 
small but nontrivial share of students (7 percent) participated in a special education program. Over one- 
third of applicants (36.9 percent) had attended more than one high school, a pattern often characteristic 
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of high household instability (Theodos, Coulton, and Budde 2014). Applicants demonstrated the intention 
of attaining a postsecondary degree, with over 90 percent indicating plans to take the SAT or ACT. 


Table 3. Academic Achievement and Educational Attributes of Urban Alliance Applicants 


Number of 
observations 

Attends charter school 

28.6% 

1,062 

Cumulative GPA as of junior year (mean) 

2.66 

966 

Enrolled in special education 

8.9% 

970 

Transferred schools 

36.9% 

866 

Has taken or plans to take SAT or ACT 

90.6% 

743 


Sources: Urban Alliance high school internship program application forms. GPAs, special education status, and some information on school 
transfers provided by DC Public Schools, Baltimore Public School Board, DC Public Charter School Board, and individual charter schools in 
DC. Variance in the number of observations stems from nonresponse for some items on the application form and forthcoming data from DC 
Public Charter School Board for program year 2012-13. 

Note: Estimates include applicants assigned to the treatment group and the control group. 


Urban Alliance applicants attended 38 different schools— a mix of public and charter schools. In DC, 
about two- thirds of youth attended one of eight schools, each accounting for 30 to 61 program applicants; 
in Baltimore, three-quarters attended one of seven schools, each serving at least nine applicants. Most of 
the schools attended in both sites are low-performing and attended mostly by minority youth (see table 
4). In fact, all schools that Urban Alliance applicants attended are majority- minority, and almost all are 
majority African American, though several in DC have significant Hispanic student contingents as well. 
About 93 percent of youth attended schools with the majority of students eligible for free or reduced- price 
school lunch. 7 

Despite these commonalities, youths' schools differ in some ways. Two schools in DC enrolling large 
numbers of Urban Alliance applicants, Dunbar High School (39 youth) and McKinley Technology High 
School (61 youth), exemplify the diversity in school characteristics. Both have over 95 percent black 
student bodies, but at Dunbar, 100 percent of students are eligible for free or reduced- price lunch and 
barely a quarter of students are proficient on District- wide reading and math exams. At McKinley Tech, 
however, just over half of students are eligible for free or reduced- price lunch and nearly 90 percent are 
proficient in math and reading. Overall, 45 percent of Urban Alliance applicants attend a school ranking 
in the bottom quartile of proficiency in reading and math in the District of Columbia or Maryland, with 
fewer than 10 percent of students at a school in the top quartile. 
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Table 4. Urban Alliance Applicants' School Characteristics (percent, 
except where noted) 

Demographics of student body for schools attended by applicants 


Percent minority 

50%-75% I 

More than 75% 99 

Percent receiving free or reduced-price lunch 

Less than 50% 7 

50%-75% 3 1 

More than 75% 62 

Assessments 
Percent proficient in math 

Less than 25% 35 

25%-50% 25 

50%-75% 3 1 

More than 75% 8 

Percent proficient in reading 

Less than 25% 22 

25%-50% 46 

50%-75% 24 

More than 75% 8 

School percentile, reading and math proficiency (in state/city) 

Less than 25% 49 

25%-50% 28 

50%-75% 1 5 

More than 75% 8 

Average size of student body 628 

Average number of Urban Alliance applicants attending school 28 

Observations (youth) 1,062 


Sources: Urban Alliance high school internship program application forms for school attended, National Center for 
Education Statistics data from 2010 for student body demographic information, Maryland State Department of Education, 
and the Washington, DC, Office of the State Superintendent of Education from 201 I for math and reading proficiency. 

Notes : Estimates include applicants assigned to the treatment group and the control group. Estimates shown are the share 
of applicants whose school falls in each quartile. The total number of schools with applicants attending was 38. 

Motivations 

Urban Alliance program participants reported in interviews that interest among their peers who are aware 
of the program is high. Unsurprisingly given the financial instability of applicants' families, many 
program staff and youth participants cited the internship wages as a major motivating factor in applying 
to the program. Some youth said they planned to use their internship earnings to help pay for college 
books and tuition. Program Coordinators felt that for most youth, the allure of high wages initially drew 
them to the program. In speaking about their decision to join the program, youth seemed to agree for the 
most part, framing the pre- work stage as an investment that would pay off in higher wages if they had the 
discipline and determination to last to the internship. 

Youth also reported interest in the experience of the program, which in most cases meant building up 
credentials on their resume for college or job applications. One youth said, "I was so excited that I got a 
chance. . . to work during the school year and have it on my resume, show something on my resume that I 
worked." Many saw the internship and training as an opportunity to develop professional and 
interpersonal skills for future use, such as knowing how to answer phones and handle other basic 
administrative tasks. For others, the internship was a chance to become more familiar with career paths 
they had not yet considered. Finally, some youth noted a prestige factor of having a professional 
internship among their social networks. 'It was diff erent, from what everyone else was doing," said one 
student. "No one's really... working or has an internship." 
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Many youth expressed a belief that Urban Alliance will help them along the path to higher education 
and careers. When asked about their post- high school plans, students who attended pre- work training 
often mentioned specific colleges and careers to which they aspired. Some looked forward to the 
additional college admissions and financial aid guidance provided by the program. As evidenced by the 
career aspirations youth noted on their applications, the Urban Alliance internship may be appealing as a 
stepping stone to various professional careers. For those students who identified a desired field on their 
applications, the six most common fields chosen all reguired two- or four-year degrees: health professions 
(identified by 23 percent); business, financial services, and management (20 percent); computer or 
mathematical sciences (8 percent); legal professions (7 percent); and architecture and engineering (6 
percent) (see table 5) . 


Table 5. Urban Alliance Applicants' Desired Future Occupational Fields 


Occupational field 

Percentage 

Health 

22.9 

Business, financial services, management 

19.6 

Computer or mathematical sciences 

8.0 

Legal professions 

6.9 

Architecture and engineering 

6.1 

Sports 

5.3 

Life, physical, or social sciences 

4.6 

Personal care and services 

4.5 

Community and social services 

3.4 

Office and administrative 

3.1 

Protective occupations 

2.8 

Other 

12.7 

Observations 

738 


Source: Urban Alliance high school internship program application forms. 

Note: Estimates include applicants assigned to the treatment group and the control group. 


Program Implementation 

The following sections on recruitment, training, internships. Program Coordinators, program attrition, 
program costs, and organizational growth detail our findings on the program's implementation. 

Recruitment and the Application Process 

Urban Alliance's role as an intermediary between schools, workplaces, and youth is illustrated in part by 
its approach to recruiting participants. In both Baltimore and Washington, DC, the recruitment process 
begins in the spring of youths' junior year of high school. Before the school year ends, program staff 
members contact counselors or other staff at schools. They remind the school contacts about the 
application process, the goals of the program, and the reguirement that youth are eligible for an early- 
release schedule to participate. 

In both sites, school contacts are informally directed to encourage the participation of youth with 
average academic performance, under the assumption that, as described by a program staffer, "below C- 
average students probably need to focus on academics and graduating high school" and "above C-average 
[students] are probably doing the million and other things you can do as a teen in DC as a senior." As 
another staffer described it, counselors encourage students to apply who have "at least a 2.5, 2.3 GPA. . hut 
as Urban Alliance has become a little bit more lax with the GPA reguirement, they have as well." One 
Program Coordinator perceives the counselors as wanting student participants who will "represent their 
schools well. . so they look for a lot of things to disqualify students" who have less than stellar grades. 

In Baltimore, as noted earlier, the program and the school system hold a formal partnership. At this 
site, school counselors reach out to specific students who they believe will be a good fit for the program. 
Counselors give youth applications to fill out and facilitate their submission to Urban Alliance. 
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One counselor reports that students usually submit their applications 'before the due date because 
they're so interested in the program. . J haven't had the problem of having to nag." Late applications are 
accepted because students often do not yet know if their senior-year schedule will allow for an early- 
release schedule. Incomplete applications are also accepted; item nonresponse was fairly high for sections 
on family members and their employment and details on youths' previous employment. 

In DC, depending on availability and school preferences, the school contact arranges for an Urban 
Alliance representative to speak at a school assembly, in a classroom, or with specific groups of 11th 
graders. Representatives may also speak informally with students individually while on site, often by 
setting up tables in the hall or the cafeteria. During presentations the representative, usually a Program 
Coordinator, explains the program, its benefits and reguirements, and the application process. Because 
the relationship between the program and the DC school system is not formal, youth either send their 
applications through the school counselors or send them directly to the program. 8 In an effort to better 
engage youth, in the 20 13 recruitment season Urban Alliance inaugurated a film they showed students on 
the program and brought in a current intern to speak to potential applicants. 

Because each youth ultimately decides whether or not to apply— and one school counselor estimates 
that half of youth who hear about the program do so— some youth who are not "middle-of-the-road" may 
apply while some who are do not. Some youth at both "the top of the class [and] the bottom of the class" 
are dissuaded by the perceived burden of filling out the application; this burden presumably decreased 
with the introduction of an electronic application for the 2013- 14 program year. Others do not apply due 
to conflicts at school or home. Some youth guestion why they do not receive wages for time spent in pre- 
work training; they are told that training is an opportunity for youth and the program to get to know each 
other, and that not until they have interviewed will they know if they have gotten the internship. 

If the school or the school counselor is new to the program, an Urban Alliance staff member meets 
with the contact to explain the program model and goals and what the program needs from schools, such 
as help arranging student schedules, copies of student transcripts during the year, and communication 
when students face school challenges. Urban Alliance also tries to gauge the school or contact person's 
needs and expectations of the program and program staff. In the 20 11- 12 program year, the DC program 
site made this process more consistent and formal in an effort to strengthen its relationships with schools. 
One Program Coordinator found that some of the counselors are "very supportive" and more engaged in 
recruitment than the students themselves; another staff member reported that "they get it, and they want 
their students to be in productive afterschool activities and self-s uffi cient." However, some counselors are 
seen as unsupportive and unresponsive to communications. In the latter cases. Program Coordinators will 
reach out more freguently during recruitment and visit them in person as needed. 

Both in DC and Baltimore, the program conducts a second round of recruitment in the fall of the new 
program year. Staff would prefer to do most or all recruitment in the spring of the previous year, but sites 
cannot always collect enough applications without a fall recruitment effort. This may be in part because 
not enough youth become aware of the program in the spring or because many are not thinking beyond 
the upcoming summer. In addition, schools and students often do not have a clear idea of class schedules 
for the upcoming school year by the preceding spring, and some students may only switch into a school 
targeted by Urban Alliance in their senior year. 

The diff erent recruiting methods between the two cities may yield different populations. DC students 
are allowed to express interest and apply, but in Baltimore, students only get referred if counselors feel 
they are appropriate for the program. However, as described earlier, we found no substantial differences 
in observable characteristics, including cumulative GPA as of the end of junior year, for youth from the 
two separate sites. 

Postsecondary and Employment Training: Pre-work and 
Workshops 

Interviews with staff and youth combined with observations of training sessions confirm that training is 
an essential element of the Urban Alliance program. In pre- work training and workshops, staff provide 
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youth with information and encouragement. They develop relationships with the youth as a group and 
have a chance to informally monitor howyouth are doing in work and life as the year progresses. Staff at 
all levels of the Urban Alliance program see pre-work as essential for preparing youth for their 
internships, as well as encouraging them to apply for and attend college. When asked what are the most 
important program elements for helping youth reach program goals, most Program Coordinators point to 
training sessions in general, and pre- work in particular. 

Format of Training 

The frequency of pre- work sessions in Baltimore and DC is similar, but the duration differs somewhat. In 
DC, there are 19 days of pre- work training; Baltimore has 21 days. In DC, there are two 75- minute back- 
to-back sessions of pre- work each day after school. Youth participate in the earlier or later session 
depending on their schedules at the beginning of the school year. At these sessions. Program Coordinators 
remind the students that they need to have their half-day schedules arranged by late October, when 
internships begin. In Baltimore, there is only one session each day during the pre- work period; it is 
scheduled for 90 minutes. In total, youth receive up to approximately 24 hours of training in DC and 32 
hours in Baltimore. 

The class size and venue used for pre- work is also quite different for DC and Baltimore, which affects 
the format of many sessions. In DC, over 150 youth begin pre- work at once in the fall. While splitting 
them into two sessions each afternoon reduces the class sizes and attrition quickly reduces it further, over 
80 youth were eligible to participate in the same pre- work session at once in early sessions. Consequently, 
most pre- work training sessions in DC were held in a large school auditorium with enough seating space 
for several hundred students. Due to youth schedules, attrition, and attendance patterns, attendance was 
usually less than anticipated, averaging about 45 youth per session according to attendance records. 

In some sessions we observed, the program staff were vigilant in making sure that each student who 
entered the auditorium sat in a tight clump near the front, eliminating empty spaces between seating 
rows. However, this was not true in all sessions. In those sessions where youth sat further apart, some 
appeared to be less engaged— they participated less frequently, did not pay attention, or discreetly played 
with their phones— or may have had trouble hearing their peers. Further, distractions and intrusive noises 
are fairly common in the auditorium. 9 Discussion in this setting was sometimes difficult, though Program 
Coordinators used vocal encouragement and eye contact to stimulate youth to answer questions and 
volunteer to read aloud from the session's materials. Some Program Coordinators engaged youth with 
jokes, personal anecdotes, or energetic call- and- response. To some degree, this also served to distinguish 
pre- work training from students' academic classes earlier in the day and to make the big group setting 
more intimate in nature. Overall, more youth seem engaged after the initial sessions as they adjusted to 
the program's format and goals— and perhaps as less motivated students left the program, a trend some 
staff and youth noted in interviews. 

In Baltimore, the program used medium- sized classrooms for pre- work training, which allowed for 
open discussion and connection between staff and youth. This was made possible by smaller enrollment 
in Baltimore, which offers only 35 job slots each year. As in DC, actual attendance was lower than 
anticipated, averaging about 30. Youth sat in desks arranged roughly in a semicircle, allowing for easy 
movement around the room during activities and work in groups. In the sessions we observed, the youth 
in Baltimore seemed to struggle more in settling down. There was somewhat continuous chatting when 
settling into the classroom and during the sessions; this contrasts with the solitary disengagement of some 
youth in DC pre- work. This may be due to the smaller setting in Baltimore and the smaller number of 
students, which could be conducive to quickly developing acquaintanceships. However, Baltimore youth, 
like DC youth, were generally fairly attentive during pre- work. Most youth appeared engaged and die 
majority participated regularly in discussions or volunteered to read from the packets provided. 

Once pre- work ended, workshop settings in the two cities were much more similar. Youth in DC were 
assigned to a Program Coordinator and spent most of the Friday workshops in medium- sized university 
classrooms with only their Program Coordinator and the other youth on that Program Coordinator's 
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caseload. There were approximately 35 youth per caseload, though the workshop sessions we observed 
tended to be smaller. Youth seemed more comfortable and alert in these smaller, classroom settings. 

In Baltimore, most Friday workshops were held in the Urban Alliance offices, which have small and 
medium- sized classrooms. Usually all participants attend the same workshop, making them somewhat 
larger than the DC workshops, which are split up by each Program Coordinator's caseload. Most youth 
participated readily in any given workshop session. 

Training Styles 

In the pre-work and workshop sessions we observed, the overall tone of the program staff was more 
casual than a teacher would employ, but more formal than a peer would use. Urban Alliance staff report 
that closeness and trust between the staff and youth is an important aspect of the program model and 
institutional culture. At both sites, youth were told to call the program staff by a title (Mr. or Ms. ) and first 
name rather than using his or her last name. We observed that the Program Coordinators made a dear 
effort to relate to and understand the youths' perspectives. In the sessions we observed. Program 
Coordinators connected examples from their own lives with the youths' experiences and challenges where 
appropriate. For example, one Program Coordinator spoke about her love of shopping when talking about 
conflicting priorities for managing money. In the majority of sessions. Program Coordinators made this 
sort of comparison at least once. 

Many Program Coordinators also j oked with the youth and used friendly teasing to express their 
displeasure with a youth's behavior if the offense was not highly disruptive to the class. Where youth were 
disruptive (e.g., chatting during the sessions or using their phones— both of which happened multiple 
times per session) or where the offense was repeated, the Program Coordinators tended to change tone 
momentarily, becoming a bit more formal. This was effective in the overwhelming majority of instances 
the research team observed. 

This workshop teaching style appeared to be important to the program's success. In the observations, 
most youth appeared to like and trust their Program Coordinators and to pay attention, even though these 
sessions involved sitting in another classroom after a long school day. Although Program Coordinators' 
relatively casual and relatable style seemed effective in keeping youth interested and in bringing lessons to 
life, there were instances where this strategy may have made the lesson less effective. At times. Program 
Coordinators connected strongly with the youth and their interests but in the process may have shifted the 
focus from the primary goals of a lesson. For example, in one DC pre- work training session on post- high 
school plans, the Program Coordinator strongly emphasized the social aspects of college, including 
parties, dating, and STDs, rather than focusing on academic ability and comfort, social supports and 
social needs, debt trade-offs, or how choice of school may help with long-term goals. 

The DC program staff dressed professionally during pre- work training— all of the men wore suits— to 
model professional behavior and appearance for the youth. Some dressed business casual once regular 
workshops began. In Baltimore, which has a smaller, more intimate program, program staff tended to 
dress business casual in both pre- work training and workshops. 

Training Methods 

In pre- work and workshop sessions, program staff used a number of teaching methods, ranging from 
lecture to student skits. The vast majority of sessions we observed included a mixture of methods. During 
pre- work, the DC staff seemed somewhat restricted by the large size of the class and the room, and tended 
to do limited group work, skits, or hands-on activities, because these involved more logistics and 
classroom management than lecture, large group discussion, individual- led exercises on paper, or 
responsive reading. In focus groups, some DC youth expressed frustration with the rigidity of the format 
during pre- work; they felt it made the sessions less interesting and engaging, and one youth even 
mentioned the discomfort of the auditorium seating. 

In general, the youth tended to be most engaged in sessions and activities that were interactive and 
concrete, such as a session on professional communication in which youth corrected and critiqued sample 
professional e-mails, as well as sessions that instructors connected concretely with real-world examples. 
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For sessions focused on workplace environment and behavior, the program sites were particularly likely 
to provide instructive examples from previous years' intern experiences. These anecdotes are part of the 
standardized curriculum that Program Coordinators use across all sites, but Program Coordinators may 
also share stories of youth with whom they have worked personally. For example, one Baltimore staff 
member urged the youth to work hard even on tasks at work that were simple or boring, citing an example 
of a past intern who had complained that her job mentor would not give her more interesting work. The 
job mentor claimed that the intern did not pay attention when doing basic tasks, such as preparing a 
batch mailing, so could not be entrusted with more complicated work. 

Training Content 

Pre-work training and Friday workshops alike addressed both soft and hard skills in the context of 
education and employment, but differed in their specific topics. Pre-work training sessions focused on 
skill- building and office behavior to prepare youth for their internships, while Friday workshops were 
more geared toward planning for future education, and developing skills needed for post- high school jobs 
or college. 

Urban Alliance staff note that many youth in the program have never been in an office setting and 
start the program not knowing howto act or what to expect in an office; therefore, training in soft skills is 
important. During pre- work training, there is a strong emphasis on "code switching ," 10 the "professional 
self," and workplace behavior, among other soft skills. Staff referred back to these concepts in many pre- 
work sessions and occasionally during Friday workshops. Youth seemed to internalize them, sometimes 
mentioning code switching specifically later on without prompting. 

The sessions on soft skills such as workplace etiquette and behavior appeared to observers to be 
among the strongest of the pre- work sessions. This material lent itself well to role-plays and discussions, 
and program staff had specific examples readily available from the experiences of previous Urban Alliance 
interns. During the year, Friday soft skills workshops focused on general life skills, self-sufficiency, and 
post- high school planning. The latter sessions included topics such as choosing the right college, applying 
to college, and applying for financial aid and scholarships. Staff repeatedly emphasized the importance of 
higher education. Although youth spend the majority of program hours at their internship, the Program 
Coordinators used the Friday workshops to keep youth motivated about higher education and view their 
work as part of the path toward it. 

Other soft skills sessions discussed managing time and money, networking, personal health, and 
relationships. In focus groups and interviews, youth indicated they got the most out of sessions focusing 
on such workplace behavior and skills, as code switching, dealing with criticism, networking, and 
professional dress. Some were confused by and disliked the inclusion of sessions on healthy living and 
relationships, wondering how these topics related to the workplace. One student reported he felt the 
discussion of domestic violence was more suited for the Maury Povich television show. 

Several training sessions focused on hard skills addressed such skills as faxing, copying, filing, 
answering phones, writing professional e- mails, and using Microsoft Word, Excel, and PowerPoint. These 
sessions appeared less highly developed and more challenging for the program to execute than those on 
soft skills. In a DC pre- work training group exercise on alphabetizing and filing, some groups were 
confused by the instructions and became frustrated with their teammates. Program staff were able to 
create from this a spontaneous lesson on how people would rather work with colleagues who were patient 
and helpful than with those who became frustrated and abrasive. 

In a different DC pre- work training session on Microsoft PowerPoint, the guest speaker mainly 
offered tips for making the presentation stand out rather than demonstrating how to execute specific 
tasks. Likewise, a Friday workshop in Baltimore on Microsoft Excel was self- directed, leading to confusion 
and distraction among the youth. Some program staff said that there was insufficient access to computers 
for teaching youth hard computer skills and letting them practice. While some youth felt they already had 
sufficient exposure to Microsoft Office products, others felt that the program had raced through the 
lessons on software and they could have used more of them. They felt that too much training time was 
spent on skills they already had, such as howto answer a phone, and not enough on new topics like Excel. 
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This sentiment aligns with reports from multiple job mentors that Excel tasks were what youth struggled 
with most during internships, and that training on the software should be increased. 

On the whole, youth articulated that they benefited from most Urban Alliance training sessions. They 
thought the training they had received would particularly benefit them in the workplace. Even one youth 
who enlisted in the military after high school felt her brief experience in pre- work training was serving her 
well there. This was especially true of Friday workshop training provided. While some youth felt the 
Friday workshops were too similar to pre- work training, most found it helpful in the end: "all of [the 
sessions] are helpful. It's just that, you know, they reiterate a lot. . and, you know, we're teenagers, it can 
get pretty boring. . But it is helping us in the long run." Explained another youth regarding some sessions, 
"I pretty much think all of us heard it before, but they just put it in more perspective. . like instead of 
telling us just teachingus howto do it." Even the participants who complained about the training 
acknowledged some benefits. Said one: "It was all very good information, but at the end of the day it is 
boring. But you know that it's. . for a good cause. . {that they're] not teaching you or showing you anything 
wrong, so you gotta listen. . if you want to make it in the corporate working world." 

Staff members are certain the training is beneficial, and several expressed the view that youth are 
often unaware of what knowledge and skills will be helpful in their later professional, educational, and 
personal development. For example, the sessions on creating and updating a resume are not very 
engaging for youth, but alumni have returned to the program and boasted that their resumes are 
formatted well and up-to-date because of what they learned in Urban Alliance. J ob mentors, too, feel that 
the training is helpful for the youth, though the mentors seemed to have only a cursory knowledge of what 
the training entailed. 

Public Speaking and the Public Speaking Challenge 

At the end of the program, during the public speaking challenge, youth give presentations to a panel of 
volunteer judges made up of community stakeholders. These PowerPoint presentations, to which youth 
add slides throughout the year, describe their internships and post- high school plans. Youth are expected 
to dress professionally and to give polished presentations. Urban Alliance provides a $ 100 bonus for high 
performance at the challenge, which 92 percent of youth participating in the challenge received. 

Both the Baltimore and DC sites used pre- work and workshop sessions as an opportunity to help 
youth feel comfortable speaking in public about themselves and ultimately to prepare them for the public 
speaking challenge at the end of the program. Starting partway through pre- work, all youth were reguired 
to create a "30 second commercial, "or elevator pitch, about themselves. At any moment during pre- work 
trainings and workshops, the Program Coordinators might call on individual youth to stand up and 
deliver his or her elevator pitch. The youth were guided to develop pitches further throughout the course 
of the program year, adding experiences or responsibilities from their internships and newly developed 
post- high school plans. During pre- work training, youth also practiced job interviewing and then gained 
experience in it through their interviews with job mentors. One youth summarized his improvement in 
public speaking as follows: '1 feel much better. . .they helped me out with my fear [of] public speaking. 

Now I feel so much more comfortable. . . Like I could do public speaking right now." As for interviewing, he 
"was shy about it, " but after receiving a perfect grade on his practice interviews during training and 
realizing he "really did a good job," he completed his internship interview, got the job, and thought, "it's 
easy now." 

Internships 

Youth who complete pre- work training enter the internship phase of the program, typically starting at the 
end of October or the beginning of November. Some interns started internships later due to Urban 
Alliance not having sufficient job sites at the time or internship slots opening up after some youth 
dropped out of or were fired from the program. Some youth were not able to obtain an early- release 
schedule until spring semester and therefore began internships in J anuary. Those unable to start the 
internship in the fall may receive a small stipend from Urban Alliance in lieu of internship wages until 
they are placed. 
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For youth that are matched with a job site in October or November, there is a kickoff event in the fall 
that Program Coordinators, youth, parents, and job mentors who are available attend. At this event. 
Program Coordinators explain to all parties the program elements, expectations and time commitment. 
They explain to parents that youth must be allowed to self- advocate and take responsibility for the 
internship. They also conduct an instructional session for job mentors, explaining to them what to expect 
from youth and program staff, the needs of youth, appropriate boundaries, and recommendations for 
downtime activities on the job. This session includes examples from previous years' experiences and 
provides an opportunity for job mentors to ask questions or discuss their own experience from previous 
years as mentors. At the end of the kick-off event. Program Coordinators introduce youth to their job 
mentors, if present. 

Described in further detail below are the process for matching youth to j ob sites and mentors for their 
internships, characteristics of the internship experience, the program's communications with job sites and 
mentors, and the value of the internship to the youth and to the sites that hire them. 

Matching Youth to Job Sites 

Youth are matched with job sites based on several factors besides availability. These include the amount 
of nurturing that Program Coordinators believe the youth will require in the job, their distance to the job 
site, career interests, and personality. Urban Alliance places a particular emphasis on level of need for 
nurturing. Program Coordinators report that some students need a lot of support and guidance from their 
mentors and, if they are matched successfully, they can perform well in their jobs. If matched with a 
mentor that is unsupportive or too busy with other work, however, they may flounder. Career interests are 
usually not a top consideration, because Urban Alliance believes other factors are more important for 
program perseverance. In addition, in interviews. Urban Alliance leadership relayed that the program 
sought to instill general rather than specific skills and was not akin to a technical training program where 
industry- specific skills are developed. 

Internship Experience 

Each job site that has agreed to partner with Urban Alliance selects the employee who will serve as the job 
mentor, guiding the youth through the internship. In interviews with multiple stakeholders, a picture 
emerged of the job mentor as a volunteer who could, but did not always, play a pivotal role in the 
program's successful delivery of services. In interviews, some job mentors truly saw themselves as 
mentors, contributing to youth development in what truly amounted to a wide- ranging mentorship role. 
Others, however, saw themselves principally as supervisors and were less willing or able to engage youth 
apart from direct work tasks. Others provided general mentoring but little job-specific guidance, mainly 
checking in with the youth about work with other employees, and perhaps providing advice related to 
general career development and education. Some mentors reported speaking with their intern every day, 
and noted that after several years they are still regularly in touch with former interns. For others, 
however, contact was weekly or monthly even while supervising their intern, if other staff at the 
organization were also involved in assigning the intern work. 

Youth work in a variety of settings, from small nonprofit organizations to large national corporations 
to government agencies, but most interns were placed in office environments. Typical job duties students 
reported performing included filing, correspondence, shipments, copying, printing, and other clerical 
duties. Some students mentioned working with Microsoft Excel or PowerPoint or entering data in some 
capacity. Administrative duties were an expected part of the internship— students spent time gaining basic 
office skills. 

Several job mentors, program staff, and students were concerned that in some cases, interns' typical 
responsibilities did not consist of meaningful work, and that job mentors did not provide interns with a 
professional experience as the program intends. Some youth mentioned having to do "busy work" and not 
gaining experience related to their desired career as problems they encountered. However, Urban Alliance 
fully intends for youth to spend some time on basic office tasks. Not only is this considered part of the 
skills growth process, but it is believed that through these tasks youth will learn such important lessons as 
persisting in a task until it is completed, being part of a work team, and understanding how each task is 


24 


URBAN INSTITUTE 


part of a larger goal. The internship is meant to provide a foundation of skills that can be used in future 
jobs in any field. 

Youth, along with Urban Alliance staff and job mentors, also relayed that sometimes they did not have 
enough work to fill their time in the internship. Urban Alliance tries to fill any work- related downtime by 
encouraging job mentors to direct youth to work on post- high school planning activities, such as updating 
their resumes or writing college essays. 

Communication with Job Sites 

Urban Alliance staff act as brokers or intermediaries between participants and the otherwise out- of- reach 
professional work environments where these youth are placed. Communication between program and job 
site staff is important to successfully facilitating that relationship. Program Coordinators and job mentors 
are introduced at the fall kickoff event and are expected to remain in touch throughout the year. Program 
Coordinators send weekly e-mails to job mentors with program updates so that mentors can be aware of 
the intern's current Urban Alliance training activities. Program Coordinators also call and e-mail mentors 
to check in on the intern's progress on an ad hoc basis, depending on how much support the intern needs; 
job mentors reported speaking with Program Coordinators about once per month. They found this to be 
sufficient, and saw more frequent communication as necessary only when a problem arose with the 
intern. The mentors generally describe these communications as helpful; one even commented that "every 
step of the way [the Program Coordinator has] been helpful and the communication has been valuable.'' 
Program Coordinators also believe that the communication is essential; they lament the difficulty in 
connecting more frequently with the mentors via phone or e-mail. 

Program Coordinators also communicate with mentors through visits to job sites. If a student is 
struggling, the Program Coordinator may conduct a "pop-up" visit, in which he or she visits the job site 
unannounced and witnesses what the environment is like for the intern. There are also scheduled site 
visits that the Program Coordinator conducts three times a year to each job site. The Program Coordinator 
speaks with the mentor and the student together and then the mentor alone. They discuss the intern's 
strengths, weaknesses, and progress. The site visit also provides an opportunity for Urban Alliance to 
build the relationship with the job mentor and job site as part of an effort to cultivate the job site to 
participate in the program again the following year. 

One job mentor described the visits positively: "I was very impressed that they're very direct. . .they 
were very to the point on addressing issues and [next steps] . . right there that showed me I like this 
program and we're going to get along. . it's not like babysitting. . all the feedback is for the student's good. " 
Another describes how after her intern had stopped coming to work and then returned, her Program 
Coordinator "came in person and met with us. She sat down with us to rework goals and an action plan, so 
she was very engaged then and that was absolutely helpful." However, another mentor recalls visits with 
the Program Coordinator lasting "two minutes," and believes that her organization has "been pretty self- 
sufficient here to manage the intern in a way that didn't have to involve Urban Alliance." It may be, then, 
that when communication is needed it is very helpful, but when it is not needed, it is seen by the mentors 
as burdensome or perfunctory. Still, Urban Alliance expects frequent communication between the mentor 
and Program Coordinator. 

Attracting and Retaining Job Sites 

Attracting organizations to serve as intern job sites involves the help of Program Coordinators, senior- 
level staff, and board members. Most job sites continue in subsequent years once they become involved 
with Urban Alliance, so attracting sites is a larger challenge than retaining them. Among the five job 
mentors we spoke with, their employers had served as Urban Alliance job sites for an average of three 
years . 11 

Program staff at all levels described a continued effort to maintain and grow the number of job sites to 
enable the program to serve additional students. For the smaller Baltimore site, this task was particularly 
challenging due to limited local visibility of the program, an employer landscape that is more constrained 
than in DC, and a thin network of professionals working to identify additional partner organizations— at 
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the time of the interview, the Baltimore "jobs board" had only a handful of active members, according to 
one local program administrator. At this site. Program Coordinators must take a more active role in 
attracting sites by spending more time networking in the community. 

Intern Value to Employers 

According to job mentors, the Urban Alliance internship program was beneficial to organizations that 
sponsor interns and to the staff that work with them directly. Organizations found that overall. Urban 
Alliance interns performed well and were helpful additions to their teams. One mentor described Urban 
Alliance's screening process as "very good," saying that it provides her organization with competent 
interns; another emphasized that her small nonprofit staff benefited from having an additional member 
on their team. Some mentors reported they desired interns with a good baseline of computer and 
communication skills; others were not concerned about skill levels at the start of the internship. Whatever 
skill level was reguired, mentors reported that hiring an Urban Alliance intern helped them meet their 
aim of increasing workplace diversity. Furthermore, the mentor position allowed managers to offer their 
employees an opportunity to supervise if they are lacking in such experience. For some organizations, the 
program was also a means of boosting image and reputation. Some reported that participating was an 
important way to be seen as giving back to local youth. 

When asked why they participate in the program, most mentors stated that their bosses requested 
they do it. However, they also described the mentoring position as rewarding. They generally enjoyed 
working with youths, and have come to view the Urban Alliance program highly. Some of the job mentors 
have a background similar to that of many Urban Alliance youth and want to help them succeed in career 
and education. 

While mentors spoke only positively of interns, there are other indicators that there were some 
negatives to their experience. Some youth reported that mentors gave them busy work and did not appear 
to trust them to competently complete tasks. Also, as discussed in more detail in the section on attrition 
below, some youth were fired from their positions. In about 22 percent of cases where interns were fired, 
the reason was an intern's poor performance. 

Internship Value to Youth 

The potential value of the Urban Alliance internship for participating youth is substantial. Participants 
have the opportunity to earn money and gain work experience in a professional setting; they may also gain 
skills and confidence. Not all youth receive all of these benefits or receive all of them equally; the 
particular gains for each individual depend on the youth's characteristics, previous experiences, 
motivation, and the job environments into which they are placed. 

At the most basic level, the internship provides paid work— and in focus groups, youth were quick to 
point to this as the major selling point for, and benefit from, participating in the program. Youth 
participated in their internships for an average of 35 weeks, working about 400 hours in all (table 6) . 

Total earnings amounted to $3,796 (in 2013 dollars) on average, with slightly more earned over the school 
year than over the summer. Youth also earned bonuses for attending events, performing at the public 
speaking challenge, referring other youth to the program, or continuing to participate in program 
activities while waiting for a job assignment. This "non- working stipend" of $25 per week may have kept 
some youth connected who otherwise would have grown frustrated with their delayed placement and left 
the program prematurely. 
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Table 6. Urban Alliance Internship Average Hours Worked and Earnings 



Minimum 

Median 

Maximum 

Weeks job held 

1 

35.0 

41 

Total hours worked 

10 

400 

781 

Hours worked through May 

0 

233 

480 

Hours worked June-August 

0 

182 

391 

Total earnings 

$87 

$3,796 

$7,9 1 5 

Total earnings through May 

$0 

$2,053 

$4,975 

Total earnings June-August 

$0 

$1,883 

$3,994 

Job earnings 

$83 

$3,653 

$7,655 

Urban Alliance incentives 

$0 

$108 

$808 

Received public speaking challenge bonus 

~ 

64% 

- 

Received public speaking challenge bonus, if attended 

- 

91% 

- 

Observations 


342 



Source: Urban Alliance financial records, service delivery data, and data from job sites that paid interns directly. 

Note: Sample includes treatment group youth who started an internship with available data on hours worked and earnings data. 


At the end of pre- work training, the program helped the youth open up a bank account for their internship 
paychecks, which are paid by direct deposit, and guided them through a budgeting exercise. During the 
2011- 12 program year, youth in DC were also offered the chance to open an individual development 
account (IDA)— a savings account that provides a 3: 1 match for savings deposits, which can be used after a 
certain period of time to purchase particular types of assets— through Capital Area Asset Builders. 
However, only 11 Urban Alliance interns opened IDAs. 

In addition to remuneration, the internship provides access to a job environment that would 
otherwise be inaccessible for most youth. Particularly given the high rates of youth unemployment, a paid 
internship for a high school student is not readily accessible. Moreover, Urban Alliance's connections 
allow many interns to work at such prestigious organizations as the World Bank and Morgan Stanley. 
Without Urban Alliance, says one youth, '1 probably would have had another job, but [not one] where I 
was making $ 10 an hour. . I probably would have been working at Wal-Mart or something." In some cases 
this access to a professional job lasts beyond the internship; some interns secure future internships or 
jobs, either at the site of their internship or elsewhere, through their Urban Alliance internship job 
mentor. Some youth also gain connections at colleges they hope to attend through their mentors. Youth 
see this as a distinguishing factor between the Urban Alliance internship and other jobs; in many cases, 
job mentors "care about [the intern's] future." 

Some youth also benefit from the support their job mentors provides. This support takes many forms, 
including tangible help with resumes, career advice, and college applications, as well as encouragement to 
complete the program, school, or internship, or navigate hardships at home. One Program Coordinator 
noted that youth with the most supportive mentors "tend to have a better wall around what's expected not 
just from the Program Coordinator. . but the mentor, the parent, everybody's kind of supporting the young 
person." Sometimes support came not from the mentor but from another employee at the job site— one 
youth did not spend much time with his mentor but felt the supervisor his job site assigned him was like a 
"second mother, really. Whenever [he] needed someone, [the supervisor was] always there." 

Beyond connections and support, the internship offers program participants work experience and 
exposure to a professional setting. One objective Urban Alliance hopes youth will achieve through the 
program is learning how to behave and succeed in an office environment. Youth, job mentors, and 
Program Coordinators all asserted that actual work experience over several months, rather than training 
alone, was essential to learning these lessons. Youth in the program recognized the value of this 
experience for college applications and for future employment opportunities. They noted that the 
internship helped them gain insights into what career path they would like to pursue, and that it helped 
them access opportunities. When deciding between one candidate who was "working at McDonald's most 
of the time" and another who completed an Urban Alliance internship, a potential employer "most likely 
would choose a guy who had office experience," said one student. 
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Participants gained a variety of hard and soft skills during their internships, according to mentors. 
Program Coordinators, and youth. Youth were less likely to mention the hard skills they gained— those 
that did spoke of gains in computer skills, customer service, filing, or written professional 
communication, and mentors also noted gains in computer skills. Both youth and mentors frequently 
reported improvements in interns' abilities in networking, office etiquette, organization, time 
management, multitasking, and preparing presentations. During one pre-work training session, students 
expressed what made them nervous about the internship: communication, "piles of paper," and finishing 
work on time. It was these skills— communicating, organizing, and managing time— that they frequently 
reported gaining from their internships. 

The improvements most commonly mentioned by youth and by staff were in communication skills, 
confidence, and comportment. This was especially evident in the focus group discussions with youth. 
During the focus group at the beginning of the program, many youth required a fair amount of 
encouragement to speak, mumbled or spoke haltingly, or made little eye contact with focus group leaders. 
In the end- of- program focus groups, many of the same youth spoke fluidly and audibly and all made eye 
contact with the focus group leaders. School counselors also pointed out improvements in soft skills, 
describing particular youth who became more punctual, responsible, communicative, or tidy throughout 
the course of the school year. However, at least one school counselor expressed concern that while youth 
may have benefited from the program, participation left youth little time to apply to colleges or apply for 
scholarships, leading them to fall behind. 

Participants reported that the internship also led to personal growth. They built confidence, social 
skills, and new relationships, and they learned to interact with diverse groups of people. Some reported 
they learned howto accept criticism better. One intern said she became more open to people through her 
internship, while another told a story of learning to speak up for herself about the type of work she wanted 
to do during her internship and how this self-advocacy made a difference. One mentor described 
encouraging the expansion of her intern's interest in the world around him by requiring him to read about 
the presidential candidates in the ongoing election, and to discuss them with her. 

While youth spoke positively about the program, according to Urban Alliance staff not all youth 
believe it is helpful. Still, there was a consensus among mentors and Program Coordinators that the 
experience was beneficial. As one Program Coordinator explained, high school students do not always 
have the best judgment of what is good for them, and staff experience suggests that the program is 
helpful. This view is supported by some of the nuance within participants' own criticisms of the program. 
In one youth's words, "I said I want to work in music and with technology, but I got placed in the office. 
And it's kind of boring, you know, I sit at my desk a lot. But you know a lot of stuff that I learned there can 
help me with what I want to do in the future." 

Program Coordinators 

The duties of an Urban Alliance Program Coordinator include running Friday workshops for their youth 
caseload; staying in touch with their caseload through a weekly group e-mail and individual contacts; 
entering tracking notes and other data into Salesforce, Urban Alliance's case management system; 
identifying resources and connecting youth to them as needed; communicating with job mentors and 
school counselors; conducting formal j ob site visits; and leading one- on- one post- high school planning 
sessions. Program Coordinators reported this set of responsibilities is both highly demanding and highly 
rewarding. 

Program Coordinators manage a caseload of 30- 35 students, a number considered ideal by Urban 
Alliance management after experimentation with diff erent, caseload sizes revealed that this size was most 
efficient given the minimal needs of some youth. Program Coordinators themselves would prefer their 
caseloads to be somewhat smaller— one offered 20- 25 students as ideal. Program Coordinators say their 
position is a lot of work, fast-paced, and often overwhelming— that it feels like "two years in one," is 
"beyond nine-to-five," and is "more than a teacher's workload." Despite these complaints, most Program 
Coordinators also describe the job as allowing for a good work- life balance. Urban Alliance is generous 
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with paid time off and flexible work schedules; two Program Coordinators say they had no difficulty 
balancing work with part-time graduate school classes. 

Post-High School Planning 

Three times a year, youth meet with Program Coordinators for one-on-one 30- 60 minute post- high 
school planning sessions. Table 7 presents the rate at which Urban Alliance interns met with their 
Program Coordinators for these sessions and how often they received help in specific areas. During the 
sessions. Program Coordinators help youth with college essays and applications for college or training 
programs, the Free Application for Federal Student Aid (FAFSA), finding and applying for scholarships, 
choosing colleges, and preparing for the public speaking challenge. The discussions become more specific 
in the spring as students narrow down the education options from which they are choosing. Program 
Coordinators may try to open students' eyes to options they had not been aware of, such as career and 
technical education degrees instead of a four-year college degree, or postponing enlisting in the army until 
after college and joining at a higher rank. Youth and Program Coordinators may also discuss issues 
related to higher education during their ad hoc communications, and these topics are also discussed 
during Friday workshops, as previously mentioned. One member of the program staff described Program 
Coordinators as "thought partners" for youth in making their post- high school decisions. 


Table 7. Urban Alliance Participants’ Post-High School Planning Activity with 
Program Coordinators 


Met with Program Coordinator 

95% 

Average number of planning sessions 

2.2 

Received help with resume 

80% 

Received help applying to college 

90% 

FAFSA 

81% 

College essay 

77% 

Student aid report review 

70% 

Observations 

343 


Source: Urban Alliance service delivery data. 

Notes: Sample includes treatment group youth who started an internship with available data. Participation rates are among youth 
who were placed in an internship, with the exception of "received help with resume" and "student aid report review," which 
were only captured for the 2012-13 program year (N = 159). 

Coaching Relationship 

The relationship Program Coordinators maintain with each intern in their caseload is an extremely 
important component of the program. Urban Alliance program managers believe that the relationship 
influences youth retention, commitment to college, and level of internship attendance. Urban Alliance 
staff said the relationship between Program Coordinators and youths is unigue to each caseload and to 
each youth. Through their passion and commitment to the program, the best Program Coordinators 
determine the needs of their caseloads and individual students and tailor their workshops and one-on-one 
meetings to those needs. Interestingly, these relationships are established practically at random, as 
Program Coordinators and interns are separately matched to job site and then paired accordingly. 

Program Coordinators strive to maintain a close, open, encouraging relationship with youth in their 
caseload. One sees her job as primarily alleviating students' stress, and another mentions the importance 
of setting an example for her students even outside the workshop, such as during a situation in which it 
was necessary to "code switch" to a more professional style of speech. During observations of training, 
there appeared to be a mutual respect between Program Coordinators and youth, which Program 
Coordinators confirmed in interviews. As one Program Coordinator described it, "you have to get to a 
place where the interns. . .talk to you about what's going on besides work. . .you can say work is great, but if 
there are things outside influencing how you do work and if you're finishing school, if you don't have that 
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relationship with the intern then you would never be able to guess. And so then you may have an intern 
that just leaves the program and you don't even know why." 

Youths agree that the Program Coordinator relationship is an important and beneficial component of 
the internship program. For example, one DC student reported a strong bond and sense of trust with her 
Program Coordinator at the end of the program, saying " I just know she's always there for me." The same 
student appreciated that her Program Coordinator allowed her to speak freely about personal issues, 
work, school, or troubles she was facing. Another student described a strong rapport with his Program 
Coordinator, although he was assigned to a new Program Coordinator after the first one left Urban 
Alliance. "I still wanted his opinion on stuff," the youth said, referring to his first Program Coordinator. 

Contact between youths and Program Coordinators varied in form and frequency. Program 
Coordinators reported connecting with youth one to five times per week outside a weekly e-mail to their 
caseload, depending on each youth's needs and the Program Coordinator's concerns. Students saw 
responding to this weekly e-mail as a hassle, so their response rates were generally not as high as Program 
Coordinators desired. Several Program Coordinators said students preferred to communicate via text 
message rather than telephone or e-mail. When Program Coordinators and youth connected, both parties 
indicated they spoke about attendance; work performance; personal, social, and family issues; academics; 
college plans; and time management. Sometimes the contact involved the Program Coordinator 
connecting participants to resources, such as child care to use while at the internship. 

Program Coordinators believe most youth need very frequent communication as part of the coaching 
relationship. While students expressed annoyance at the high level of contact from the program — " They 
keeping calling us like back to back to back to back. . .They e-mail you and text you and text you," "they 
have robo calls, keeping calling until you pick up."— they also seemed to be grateful for it. As one student 
explained, "even if it was nagging it's a good nagging. You know howyou get a good nagging when. . it's 
something that's going to help you in the long run. So if it is nagging, they did a pretty good job." After 
program completion, youth tend to reach out to their former Program Coordinators rather than alumni 
services, which speaks to the closeness of the bond that develops. 

Despite evidence of the strength of the coaching relationship. Program Coordinators reported several 
challenges in working with youth. These included difficulty managing a large caseload of interns; 
insufficient time to manage students' professional, academic, and post- high school goals; and a lack of 
resources and training to guide students through serious issues in their home or personal lives when 
necessary. 

Many Program Coordinators, especially those in DC, cited the same primary concern: that their 
caseloads were too large for them to provide needed one-on-one support to all youth, especially for post- 
high school plans. Program Coordinators described problems caused by the large caseload, including 
disproportionate attention given to students with the greatest needs. "You tend to gravitate towards the 
students that are ranked high as far as what their crisis is," one Program Coordinator noted. Another 
mentioned that because of the large caseload and the administrative responsibilities linked to each intern, 
it was not possible to meet in person with all of her students' job mentors, or visit them at their job sites as 
much as would be helpful. Other Program Coordinators noted that the stress of the high caseload was 
exhausting and could "bum you out." 

Many Program Coordinators and at least one program administrator acknowledged a tension in the 
Program Coordinator role between the amount of time devoted to encouraging youths' professional 
development and the amount of time spent supporting their academic success. Although Program 
Coordinators conveyed that both were vital to Urban Alliance's mission and of great concern to funders, 
they viewed professional and academic oversight as competing with one another given time constraints. 
'With these two goals in mind we have brought up the question... which one is priority, and I don't know if 
we have an answer for that because in order to be a professional, we want you to have those post- high 
school plans, " one coordinator said. Another Program Coordinator, describing upcoming one- on- one 
meetings with his interns, was overwhelmed by the ground to be covered in each session, including 
discussion of the public speaking challenge, post- high school planning, grades, and professional 
development. Several Program Coordinators felt that adding a position dedicated to post- high school 


30 


URBAN INSTITUTE 


planning would alleviate the competition between Program Coordinator priorities of professional and 
academic growth. A member of the senior leadership, however, felt it was important for Program 
Coordinators to work with participants on post- high school planning because they had a deeper personal 
knowledge of the youth. 

Program Coordinators also felt constrained in their ability to help youth facing serious problems in 
their personal lives. Program Coordinators felt that they had very limited tools at their disposal — 
essentially a binder of referral resources and the ad hoc advice of their coworkers— to help students in 
need of housing, family counseling, or other forms of emergency assistance. While asserting that Urban 
Alliance runs an effective program for all youth regardless of the challenges they face, job mentors agreed 
that tools were limited, noting that program staff could do more to support youth with extraordinary 
needs outside of the program. One job mentor recalled a participant who struggled to complete the 
program because of domestic violence; the mentor felt there was not a sufficient intervention on this 
youth's behalf. 

Program Coordinators noted the lack of training they have in dealing with these matters. One 
Program Coordinator felt that "crisis management training would be very helpful" because supporting 
students undergoing emotional trauma could be "overwhelming" for her and her colleagues. Another 
Program Coordinator agreed that the lack of training was a problem, and that senior staff were also not 
able to provide concrete advice on howto "support a young person... with actual tangible services." One 
senior staffer agreed that Urban Alliance needs to invest more in Program Coordinator professional 
development, describing the Program Coordinator role as "a jack of all trades, a master of absolutely 
none," and that the job requires knowledge of "social work, mental health, sexual health.. .all of it to serve 
our young people appropriately." Besides additional training for Program Coordinators to support 
students with extraordinary needs, some staff stressed the need to build Urban Alliance's relationships 
with other service- providing organizations so that students could be quickly connected with the resources 
they need. One Program Coordinator mentioned a successful partnership with a nonprofit organization 
providing professional attire to low- income workers and felt that Urban Alliance should develop similar 
ties with housing, family counseling, and other nonprofit groups that could provide needed services to 
interns. 

Communication with Schools 

Urban Alliance has recently invested in streamlined communication with schools. Historically, 
communications between Urban Alliance and school staff were ad hoc and fragmented. With multiple 
Program Coordinators serving youth from each school, a counselor could have multiple points of contact 
within the organization, especially for the DC site, which is larger than the Baltimore site. Urban Alliance 
program staff perceived that this caused inefficiencies and missed opportunities. During the 2011- 12 
program year, the DC program developed a new system for communication with individual schools, 
assigning a specific program staff member as the main point of contact with each school. Program staff 
and school counselors with whom we spoke reported that this change clarified all aspects of program 
involvement with the schools and improved the relationships between the schools and the program. In 
this new model, if a Program Coordinator has a question or concern pertaining to a particular youth, he or 
she may ask the program staff member assigned to that youth's school to reach out to the school 
counselor. 

In both DC and Baltimore, program staff and school counselors report that they were in contact with 
one another for a variety of reasons. Program staff contact the school to provide updates and compare 
notes. For example, if a student is having trouble at home and has a dose relationship with the school 
counselor, the Program Coordinator may contact the school to ensure that the student is receiving 
adequate support from all avenues. School counselors sometimes contact Program Coordinators if they 
are concerned about a change in a youth's attendance, academic achievement or post- high school plans. 

If a student's grades drop too low and do not improve after a probationary period, the student will be 
terminated from the Urban Alliance program to focus on school. On occasion. Urban Alliance has 
coordinated with a school to provide a youth with time off from the program without termination in order 
to focus on a home or school issue. 


Preparing Youth for College and Career: A Process Evaluation of Urban Alliance 


31 


The frequency of communication with schools varies depending on the style of the school counselor 
and the challenges faced by particular Urban Alliance interns. Program staff noted that some school 
counselors are in touch with them regularly to provide and receive updates on their students; others only 
get in touch if they notice a specific problem. Urban Alliance staff contact school counselors for occasional 
updates, but are primarily in touch when they have a concern about a student or when they want to alert a 
school counselor to a change in a student's status, such as a student being placed in an internship or 
leaving the program. This is important to school staff because they cannot permit a half-day schedule for a 
student who is no longer in the program. 

School counselors reflect positively on their communication with schools. They say "communication is 
great between the program and the schools, " that Urban Alliance staff 'let me know if there's a problem, " 
and that staff are "really good" about giving updates on youth. Likewise, Program Coordinators feel the 
schools are "incredibly supportive," communication is "great" and that communication makes a difference 
in youth outcomes. They also acknowledge the relationships can be delicate because of the counselors' 
important role in the recruitment of youth; "we don't want to frustrate them cause they're our 
gatekeepers." Still, both staff and counselors are overwhelmingly positive about their communication. 

Qualifications, Training, and Retention 

Most Program Coordinators enter their role with some sort of postgraduate degree, commonly a master's 
in counseling or education, and all have at minimum a bachelor's degree. They tend to have a few years of 
experience working in a nonprofit or public- sector setting, often in a mentoring or teaching role. Three 
Program Coordinators had worked at Big Brother Big Sisters of America; others had worked at the YMCA, 
a public after- school program, a high school violence prevention program, a youth entrepreneurship 
nonprofit, and as a social worker at a group home. Urban Alliance prefers Program Coordinators to have 
about two years of direct service experience before starting. Without this, senior staff feel that Program 
Coordinators are likely to have difficulty managing a large, diverse caseload along with all the other 
aspects of the job. Although all Program Coordinators will encounter situations with which they do not 
have previous experience, it is best if they have developed skills in time management, administrative 
tasks, and basic office behavior. An Urban Alliance management staff member describes the Program 
Coordinators as a mix of young, energetic, less- experienced people with older, more experienced people. 
While older staff may have the advantage of experience, younger staff may be better able to relate to the 
youth. Management believes that the ideal Program Coordinator is somewhere in between these two 
types, but a mix of the two types among staff also works well, according to the management staff. Program 
Coordinators work together closely and reported sharing guidance and dilemmas about youth in their 
caseloads with one another frequently. 

Incoming Program Coordinators complete a week-long orientation, though many did not include this 
in their description of the training they receive at Urban Alliance when we inquired during interviews. 
Program Coordinators receive a program manual that includes a section on the Urban Alliance culture. 
They can opt to shadow a current Program Coordinator on a job site visit, though again. Program 
Coordinators did not mention this opportunity and may not have been aware of it. Each Program 
Coordinator is also allotted a $250 stipend for further professional development. Some Program 
Coordinators said they had not taken advantage of this, but a few had been to events on financial aid, 
financial literacy, or youth development. 

Program Coordinators said that they experience a large amount of on-the-job learning during the first 
year in the position. Through a process of trial and error, they learned howto deal with a variety of 
situations and different types of students. They also said they received valuable support from upper- level 
staff and from their peers. Program Coordinators are supported by a program manager for their specific 
location, by the organization's chief program officer, and by other head office staff as needed. One 
Program Coordinator mentioned seeking advice from her superiors on time management because the 
orientation they receive focused more on case management. Generally, Program Coordinators conceded it 
was hard to train someone for their position. If there was room for more training, they would ask for it to 
focus on howto handle specific situations with a student, howto counsel students, howto provide youth 
with services, crisis management, and establishing consequences for youth when necessary. 
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The flexible and collaborative Urban Alliance culture may encourage Program Coordinators to stay at 
Urban Alliance, but the high intensity and limited salary of the position results in a typical duration of two 
to three years. Occasionally it has been much shorter because poor performance or abuses of Urban 
Alliance's flexible work schedule have led to staff terminations. Like many direct service positions, the 
Program Coordinator role can lead staff to feel burned out. About 70 percent of the Program Coordinators 
with whom we spoke were in their first or second year as a Program Coordinator. That said, some 
Program Coordinators do remain at the Urban Alliance and advance to the management level. Urban 
Alliance prefers to promote from within its ranks to provide a route for advancement for talented staff and 
to help preserve its organizational culture. Five of the senior staff members are former Program 
Coordinators. With the frequent turnover of Program Coordinators, this practice allows institutional 
knowledge to build rather than disappearing with each cohort of Program Coordinators. Still, 
management is considering trying to make the Program Coordinator position more amenable to a longer 
tenure, because they believe more experience makes for a better Program Coordinator. 

Program Attrition 

Substantial attrition occurs in the Urban Alliance internship program, both before and during pre-work 
training and, to a lesser degree, throughout the year (table 8). Of youth who completed an application and 
were admitted to the treatment group, more than one in five (22 percent) did not show up to pre-work 
training. Approximately another quarter of accepted applicants (26 percent) did not complete pre-work 
training, about one- third of those who began pre-work training. Of the 52 percent of accepted applicants 
who completed pre-work training, the vast majority (93 percent) were placed in an internship. 12 Due to 
some further attrition during the internship phase, 41 percent of accepted applicants ultimately completed 
the program. 13 Table 8 also shows the relative attrition rates of youth beginning pre- work training (second 
column) and of youth placed in an internship (third column) . Of those who began pre- work, a little more 
than half completed the program, while 84 percent of youth who began an internship completed the 
program. 

Urban Alliance and its stakeholders do not necessarily see high rates of attrition before and during 
pre- work training as problematic. Staff, school counselors, and even youth reported that they view pre- 
work training as a way for youth to demonstrate a commitment to the program and to their prospective 
internships, ensuring that students with low motivation or irreconcilable scheduling conflicts leave the 
program before, not after, beginning an internship. Furthermore, attrition before the internship stage 
does not result in internship opportunities going unfilled, because there are consistently more students 
completing pre- work training than available placements. Of course, high rates of attrition raise the 
question of whether the program substantially benefits youth or rather acts as a screening mechanism for 
identifying the more motivated youth who would have done well even in the program's absence. The 
randomized controlled trial design will help inform this question in subsequent evaluation reports. 


Table 8. Youth Participation in Urban Alliance Internship 
Program Components (percent) 


Application accepted 

Began pre-work 

Placed in job 

Began pre-work training 

77.7 

- 

- 

Completed pre-work training 

52.3 

67.3 

- 

Placed in job 

48.6 

62.5 

— 

Completed program 

40.9 

52.6 

84.1 

Observations 

700 

544 

343 


Source: Urban Alliance service delivery data. 

Note: Participation rates for applicants are among youth invited to join the program (the treatment group). 


Although Urban Alliance views attrition as an intentional, functional feature of program design, some 
of the factors contributing to attrition are potential challenges that Urban Alliance may want to address. 
The driver of attrition most commonly mentioned by youth and Program Coordinators was the cost of 
transportation to and from training events and job sites. While this issue contributes to attrition primarily 
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before the internship phase of the program, before attrition is viewed as a problem, it remains a 
significant financial and scheduling difficulty for many students throughout the length of their 
internships. Program staff were aware of the problem, with one Program Coordinator from DC noting that 
the issue continued even when interns have the opportunity to earn wages: "Once you add the internship 
experience and they have to get somewhere, they just can't finance it." Students brought up similar 
concerns, and one noted that a job mentor offered assistance in covering costs of transportation but never 
actually provided any resources. While there is some financial assistance or discounting for transportation 
available to youth during the school year. Program Coordinators and students agreed that program 
participants were largely on their own to deal with transportation financing, and some assumed that 
students who were committed to attending the program would find a way to overcome this obstacle. Many 
youth could benefit from a form of transit voucher during the summer stage of the internship as well. 

The scheduling demands of participating, too, kept many prospective interns from continuing the 
program, especially in the pre- work training phase. The program places substantial demands on youths' 
time, requiring that youth participate in work or workshop from 2 : 00 p.m. to 5: 00 p.m. each day during 
the school year. Though Urban Alliance considers job site location and distance from school when placing 
a student, some youth who did not complete the program cited long transit times as their reason for 
quitting, as well as competing priorities such as athletics and other after-school activities. These 
extracurricular scheduling conflicts often became clear very early into program participation, contributing 
significantly to attrition in the pre- work training phase. 

For some students, school class schedules were an impediment to participation and led to attrition 
primarily in the pre- work training phase. Although in recent years Urban Alliance has attempted to verify 
school schedules earlier in the recruitment and application process to be certain youth are eligible, 
conflicts still arise. Even once the school year begins, many schools are unsure of youth eligibility for an 
early- release schedule. In some instances, youth are invited to pre- work training but are ultimately unable 
to have their schedules rearranged to participate in an internship. 

Other youth could alter their schedules but opt not to. According to one Baltimore school counselor, 
these youth become "comfortable in their schedule" of full school days followed by pre- work training, and 
when the time comes to begin an internship and switch to an early- release schedule, they choose to retain 
their current schedule and decline the internship. Program staff noted that, when arranging youth 
schedules, it is important that they have good relationships and clear communication with the school 
contacts. In some cases. Urban Alliance is able to make slight adjustments to its requirements to 
circumvent school scheduling co nfli cts. For example, a student may arrive at work later than 2:30 on 
some days if the arrangement is acceptable to the student's job site. Generally, the scheduling conflicts 
stem from youth needing to take a specific course rather than from needing a higher number of credits; 
Baltimore school counselors noted that most of their youth could have participated even without receiving 
course credit for the program. 

Other causes of attrition during pre- work training reported by youth include a lack of motivation or 
interest to complete training, personal or family issues preventing continued participation, and relocation. 
While most attrition occurs during pre- work and before the internship phase, there is a substantial drop- 
off rate during internships as well. Of the 343 youth in the study who began an internship, a total of 57 
youth did not complete the program. Among those for whom the reason for departure is known, 20 quit 
and 31 were fired. Of those who were fired, the primary reason was usually attendance ( 15 youth) or 
misconduct ( 10 youth), which frequently involved time- sheet fraud. Five were fired for poor performance, 
and one was asked to leave because the program was interfering with her academics. Among those who 
quit, common reasons were wanting to pursue another job or educational opportunity (6 youth), having a 
personal or family obligation that took precedence over the program (6 youth), needing more time for 
high school studies (4 youth), and disliking the internship (4 youth). 

Program Costs 

Urban Alliance makes a significant financial investment in program participants through staff time 
dedicated to individual mentoring and training, special events celebrating the accomplishments of interns 
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at year's end, and wages paid for a work experience of several months. Although this baseline report 
cannot quantify program outcomes monetarily, financial information provided by Urban Alliance allows 
for a detailed assessment of program costs. The approach to this assessment is to include only expenses 
that are directly needed to maintain the internship program in Baltimore and DC, excluding costs related 
to other programs run by Urban Alliance as well as organizational— but not clearly programmatic— efforts 
related to fundraising, administrative duties, and expansion to other sites. In this report, we present 
information about the total and per- student costs of implementing the Urban Alliance high school 
internship program to inform the broader youth development and youth employment fields, should other 
nonprofit organizations wish to adopt a similar intervention. 

In fiscal year 2012 (J anuaiy to December), Urban Alliance's total expenses were $3.77 million (in 
2013 dollars, adjusted for inflation), with about half of that amount supporting the Baltimore and DC 
internship programs. The organization employed 41 full-time paid staff members and one part-time paid 
staff member. About 195 volunteers assisted at different events, and 277 professionals served as job 
mentors, free of charge. Table 9 summarizes Urban Alliance internship program costs for fiscal year 2012 
in Baltimore and DC. Youth participant wages, participation awards, and fringe costs (employer 
contributions to Social Security, Medicare, worker's compensation, and unemployment benefits— but not 
medical, 40 l(k), or other benefits) together made up nearly half (46 percent) of all program costs. Staff 
wages and fringe benefits made up another 42 percent of program costs. Rent (6 percent) accounted for 
surprisingly little of the program costs, though senior staff expect this share to rise in later years as the 
organization has moved to a larger and higher- rent facility. 


Table 9. Urban Alliance Internship Program Costs, Fiscal Year 2012 


Cost category 

Amount 

Percent of total 

Student wages, awards, fringe 

$856,244 

46 

Direct labor 

$654,125 

35 

Staff fringe 

$132,260 

7 

Rent 

$1 12,554 

6 

Miscellaneous administrative costs 

$99,084 

5 

Other direct program costs 

$25,337 

1 

Total 

$1,879,603 

100 


Source: Urban Alliance fiscal year 2012 audited financial records. 

Note: In-kind donations for rent and consulting services, totaling $71,778, are included in the rent, miscellaneous administrative 
costs, and total costs. 


Table 10 presents a breakdown of Urban Alliance's spending on staff wages and fringe labor costs for the 
Baltimore and DC internship programs ($786,385 total) into spending on various types of staff. Nearly 
three-quarters (74 percent) of the total was devoted to Program Coordinators and other site- specific staff; 
the remaining 26 percent supported national office staff such as the chief executive officer, chief operating 
officer, director of employer partnerships, and director of evaluation. The DC site used substantially more 
staff wage resources than the Baltimore site, reflecting the larger number of students it serves. 


Table 10. Urban Alliance Internship Program Allocation of 
Spending on Personnel 


Position 

Percent of spending 

Site staff (Baltimore and Washington, DC) 

74 

Program Coordinators 

50 

Other site-specific staff 

24 

National office staff 

26 

Total 

100 


Source: Urban Alliance 2012 audited financial records. 

Notes: Estimates include only time spent on the Baltimore and Washington, DC, internship programs, weighted to account 
for wage differences among staff. Other site-specific staff include program managers and directors, the Baltimore site 
executive director, and the Baltimore career counselor positions. 
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Given significant attrition from pre- work training to the internship phase, the cost per participant 
depends on the definition of participation. Dividing the total program cost by the number of youth who 
began pre- work training results in a low cost per student and follows the methodology of many cost 
studies that count all individuals who participate to any degree in a program. However, this data point is 
not the most informative as many youth do not persist far into pre- work training or receive any internship 
wages, which make up the largest segment of program costs. The cost per student of those who complete 
an internship is higher, but using completion as the criterion for inclusion ignores students who leave the 
program prematurely but may still gain meaningful experience and skills from the services they received. 

Table 11 presents estimates of the cost per student using four possible denominators. When dividing 
program costs across all youth who began pre- work, the cost per student is $4,925, with $2,243 for youth 
wages (though many of these students leave the program before ever receiving a payment) and $2,681 for 
program staff, rent, and administration. When using the much lower denominator of youth who 
completed pre- work, the cost per student is $ 7, 3 14, with $ 3, 332 going directly to youth and $ 3, 982 for 
program staff, rent, and administration. Dividing program costs only among those youth who completed 
the program yields a cost per student of $8,866, with $4,039 for payments to youth and $4,827 for 
program staff, rent, and administration. 


Table I I. Baltimore and DC Internship Program Costs per Youth, 
201 1-12 (2013 dollars) 



Began 

pre-work 

Completed 

pre-work 

Placed in 
internship 

Completed 

program 

Rent, staff wages, and 
administrative costs 

$2,681 

$3,982 

$4,1 10 

$4,827 

Wages and awards 

$2,243 

$3,332 

$3,439 

$4,039 

Total costs 

$4,925 

$7,314 

$7,549 

$8,866 

Number of students 

548 

369 

343 

289 


Sources: Urban Alliance 2012 audited financial records and service delivery records. 

Notes : Cost per student is an estimate using number of students from the 201 1 — 12 program year, which runs from 
September to August, and total program costs from 2012 fiscal year, which aligns with calendar year. Wages includes 
wages and associated fringe. Numbers of students are of youth assigned to the treatment group. 


In examining other youth development, training, and college preparation interventions, costs for the 
Urban Alliance program are generally in line with those of other efforts, though wide variation in program 
components makes drawing comparisons difficult. Some programs, such as those that offer housing, have 
costs per person more than double that of Urban Alliance ( Schochet et al. 2006) . Other programs more 
similar to Urban Alliance have reported more comparable costs. Upward Bound, which does not include a 
job component, cost $4,800 per year in 2001 (or $6,314 in 2013 dollars; Myers et al. 2004) and After 
School Matters, which includes an apprenticeship with much lower pay than the Urban Alliance 
internship, cost $2,520 per participant in the 2003- 04 school year ($3,108 in 2013 dollars; Proscio and 
Whiting 2004). 

As mentioned above, the upcoming impact study will include an analysis of these costs in light of 
program outcomes and an estimate of the program's overall cost effectiveness. 

Organizational Growth and Change 

Urban Alliance has experienced tremendous growth over the past decade, transforming itself from a local 
nonprofit with four staff members to a multisite organization with 42 staff members that has served over 
1,500 students. The Urban Alliance high school internship program now operates in four sites, having 
expanded from Washington, DC, to Baltimore in 2008, to Chicago in 2012, and to Northern Virginia in 
2013. 
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Urban Alliance now proactively looks to join new markets, though in the case of its first expansion, to 
Baltimore, it was the school system that sought out the organization for a partnership. Senior staff report 
that Urban Alliance will enter a market if it believes there is a need in that locality, at least 70 internships 
can be secured, the regional staffing can be put in place, and senior leadership agrees on the decision. 

Also, and perhaps most important, staff must feel that there is buy-in from the local community and local 
corporations. A corporate "champion" can help Urban Alliance make the necessary connections in the 
business community so that they can line up jobs for students. Urban Alliance senior staff report that it is 
important for business leaders in a community to buy into the idea that Urban Alliance students are their 
future workforce and that they can help prepare them for jobs that will stay in their communities. 

Staff members say it can be challenging to navigate the program model in each site. The newly created 
chief program officer position was designed to ensure that the sites implement the curriculum and share 
information in a standardized manner. The DC site has been able to share resources and information with 
the Baltimore site due to their proximity. Still, communication between Program Coordinators in DC and 
Baltimore occurs primarily through Program Coordinators from one site attending major program events 
at the other site, which are infrequent. 

Growth within a particular Urban Alliance site is dependent on the number of local internship slots 
that can be secured for students. The difficulty of this task varies across the sites. The Washington, DC, 
site has had an easier time securingjob sites because of a more robust job market and the concentration 
of government agencies, as well as corporate members of its board who advocate for the program. The 
Baltimore site has struggled to secure more job sites and has created its own board of directors to help 
cultivate them. As staff work to increase the supply of job sites, they report finding that student interest 
also increases over time. They note that participants' classmates see them dressed professionally for a 
paid, prestigious job and are inspired to join the program when they become seniors. However, the 
organization's leadership asserted that they do not want any site to grow too large, because they believe 
very large sites would deliver a less intensive intervention and would result in lowered staff functioning 
and a weaker organizational culture. 

In addition to an expansion in the number of students at each site. Urban Alliance has experienced 
growth in the technologies and tools that each site uses to serve its youth. In 20 12, sites began to use 
Salesforce's web- based case management system to electronically track youths' characteristics and 
program participation. In this system. Program Coordinators track youth attendance at program events, 
workshops, and internships; enter notes about their contacts with youth; and record youth progress 
applying for college, financial aid, and scholarships. Program Coordinators and their supervisors say that 
the Salesforce platform has helped them better track and serve their substantial caseloads of youth. Site- 
specific tools include laptops for students to use in the Urban Alliance office in DC. In Chicago, a national 
staff member said "the sky is the limit" when it comes to accessing exciting speakers and venues for 
events. For instance, youth participated in a healthy living training with Chicago Bulls trainers at the 
Bulls' training facility, and Michelle Obama recently visited program youth for a question- and- answer 
session. 

This extensive growth within sites and in the number of sites has required a major staff reorganization 
and expansion. Each site now has its own executive director and program director or manager. The 
number of executive staff at Urban Alliance, all of whom are located in Washington, DC, has increased 
significantly. The executive staff includes newly created positions of chief program officer, chief 
development officer, chief operating officer, director of alumni services, director of evaluation, and chief 
of strategic partnerships. These staff members' roles are intended to allow the organization to function 
consistently in multiple sites and permit the Program Coordinators to spend their time serving students 
instead of cultivating donors and partners. 

Two of these positions, director of alumni services and director of evaluation, are the result of Urban 
Alliance developing new initiatives in these areas. From 2006 through 20 10, Urban Alliance partnered 
with the Urban Institute and the World Bank in an evaluation- capacity building project called the East of 
the River Initiative, which elevated Urban Alliance's capacity and interest in evaluation. In 2011, inspired 
by the need to keep in touch with program graduates for research purposes, the organization added an 
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alumni services component to its internship program. Previously, formal alumni outreach and support did 
not extend beyond hosting an alumni panel during one school- year workshop and an alumni reunion. 

In 20 11, the programming for alumni was enhanced to include individual outreach, more events for 
alumni, and an alumni ambassador program (peer mentoring) . Urban Alliance offers one-on-one 
coaching meetings for alumni who contact them, help with transferring between colleges, help with 
renewing financial aid, support redesigning a resume, or continued training in workplace skills. 
Additionally, Urban Alliance created a resource room which includes college and job- seeking brochures 
and computers with internet access available to alumni. Urban Alliance remains in touch with alumni and 
posts job announcements through a networking website it hosts as well as such existing social networking 
services as Facebook, Linked- in, YouTube, and Twitter. 

Urban Alliance also now offers internships for college- age youth. In DC, there is a college summer 
internship program for alumni; in Baltimore, youth are connected with jobs through a local youth 
summer- employment program. All of these services for alumni are intended both for college- enrolled 
youth as well as youth who have become disconnected from school or work and have limited avenues for 
reestablishing those connections. 

Along with the enhanced activities for alumni. Urban Alliance has placed an increased emphasis on 
the tracking of alumni and evaluation activities in general, with the oversight of the newly created director 
of evaluation position. Program Coordinators use Salesforce to enter data on both current participants 
and alumni that can be used in outcome tracking. Urban Alliance accesses data from the National Student 
Clearinghouse to determine where and when alumni have enrolled in college. Students who cannot be 
located in National Student Clearinghouse are called individually to ascertain their post- high school 
education or employment. Urban Alliance asks youth to fill out surveys and participate in focus groups 
about the program, which it uses to ensure that students are engaged and to tailor curriculum and services 
where possible. It has also developed a skills growth survey for job mentors to fill out about their interns, 
to determine what skills youth are gaining through the program. 

This evidence of the Urban Alliance program's growth in recent years matches staff member's 
perceptions of the organization's approach to growth and change. Staff say Urban Alliance has a 
nonnegotiable core essence, but is open to feedback. The organization will change details as needed and 
even enter new realms such as policy and advocacy, but staff also express a desire not to grow too large 
too quickly. Senior leadership solicit internal suggestions for changes through the annual planning 
process, short-term planning with individual staff members, quarterly meetings, and external suggestions 
through surveys and focus groups. Senior staff report they work to keep the Urban Alliance culture intact 
because they see it as a key incentive when recruiting new staff and "one of the main reasons why people 
stay" for multiple years. 

Implications for Practice and Policy 

The Urban Alliance High School Internship program focuses on an important segment of youth at a key 
turning point in their progression into adulthood. Although most students who enter the program would 
likely graduate high school in the program's absence, these youth are at risk of becoming disconnected 
from both school and work upon graduation. Few programs have proven successful at helping 
disconnected youth; the Urban Alliance program serves to orient youth toward college or employment 
before completing high school with the aim of preventing disconnection and supporting self-sufficiency. 

The program fills an important niche in the continuum of services provided to at-risk youth. Low- 
achieving students who need significant remedial support generally are not able to participate in the 
program; nor are high- achieving students, due to full schedules that prevent early release. Therefore, the 
program essentially targets "middle-of-the-road" students who are likely to graduate, but who may have 
difficulties acquiring good jobs after high school. Intervening with these at-risk youth while they are still 
in high school provides the opportunity to enhance youths' post- high school education and employment 
outcomes. 
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Efforts to link students with the world of work date back many decades including major initiatives 
such as the National School-to-Work Office (a joint endeavor of the US departments of Education and 
Labor in the late 1990s), career academies, and career magnet schools. The Urban Alliance model has 
significant implications for serving at- risk youth in this capacity. Urban Alliance has created an 
intermediary role between schools and employers, relieving schools of a task for which they may be ill- 
suited. Employers are able to deal directly with a responsive organization that will provide interns with a 
beginning set of both hard and soft skills. 

The intermediary function Urban Alliance provides does not relieve schools or employers of certain 
responsibilities important to the success of the program. During the school year, students work at their 
internships in the afternoon and must able to leave school early. School administrators must buy in to the 
Urban Alliance model to support students trying to complete their schooling and gain work experience 
concurrently. School counselors and other officials play an important role in identifying students who can 
best benefit from the Urban Alliance program and perform well enough in school that they can be allowed 
to leave early during the day. The Baltimore and Chicago school systems offer course credit for the Urban 
Alliance program to encourage youth to participate, a feature other localities could consider. 

Employers, too, share responsibility for the success of the program. They must be willing to welcome 
low- skilled high school students and give them genuine work opportunities in an office setting. Given the 
backgrounds of many of these youths, employers must understand many will be "rough around the 
edges." Rather than guickly dismissing a youth for certain behaviors, the employer needs to be willing to 
negotiate youths' issues with the help of Urban Alliance. To make the experience successful for the youth, 
employers must provide mentorship and feedback. This requires a dedication to the principle of the 
program that can be somewhat demanding of the mentoring staff. 

The Urban Alliance model includes both classroom- based and experiential learning, incorporating 
several aspects of youth employment programs many consider important, such as paid work experience, 
training on soft skills, and alumni services. The program also offers mentoring in two contexts, within the 
program setting and on the job. Sometimes called the "caring adult" model, many believe that a key to 
success for at- risk youth is having a caring adult in their lives who can help guide them. 1 mportantly, the 
mentoring literature indicates these relationships must be long-term (at least one year) to have an impact, 
allowing the youth time to establish a genuine relationship. 

Each program component contributes to a youth's future labor market success. However, Urban 
Alliance's major emphasis for what participants should do after high school centers around giving youth 
an appreciation for the value of attending college. The returns to college over high school completion have 
been increasing and are well- documented. 14 However, disadvantaged youth who can benefit from college 
frequently do not attend as they do not understand its value or believe they cannot afford the cost of 
attendance. Urban Alliance provides these youth with office- based work experience that helps them 
understand how college can improve their future opportunities and earnings. One weakness cited by 
program participants is a lack of adequate connection to postsecondary institutions. If college is the goal, 
students feel more could be done to introduce students to college, such as sponsoring campus visits. 

Although heavily aimed at encouraging college enrollment. Urban Alliance recognizes that college is 
not the answer for all youth. Through its pre- work training, workshops, and the internship. Urban 
Alliance gives youth job- readiness skills that will help them obtain employment immediately if that is 
what they choose. However, given that the internships are geared toward employers who hire more 
educated workers, good performance in their internship alone is unlikely to lead directly to a job with the 
intern's employer. The tasks youth perform during their internships do not prepare them for well- paying 
jobs upon high school completion, but the demonstration of soft skills and work experience should put 
them ahead of other students not bound for college. 

The Urban Alliance funding scheme is an important component of the Urban Alliance model. By 
having employers pay Urban Alliance a contribution. Urban Alliance can oversee the payment of wages to 
youth and cover the additional costs of administering their high school internship program, allowing the 
organization to achieve sustainability of the program after initial start-up. Philanthropic funders can 
provide the initial start-up funding for a new school district, knowing their contribution can be limited. 
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Such funders can help launch a sustainable program, providing significant impact from their investment. 
On the other hand, a minimum number of employers must be engaged to make the program viable at a 
site. In many cases, a local champion with connections to the local schools and local employer community 
may be needed. To expand broadly and be sustainable requires hiring solid local program directors. Once 
the program is in place, good performance should lead to sustainability within the site. Sustainability is 
also related to scale. Questions about the appropriate scale for Urban Alliance will be resolved by the 
program going forward, in terms of both the number of cities that could support the intervention and the 
number of youth in a city that could participate in it. 

The Urban Alliance program relies on youth to self- select for entry into the program and continuation 
in it. Urban Alliance wants youth who are motivated to engage in the pre- work training, internship, and 
workshops. This condition is critical for keeping employers engaged and willing to participate in the 
program. Significant attrition occurs among those who apply, requiring Urban Alliance to recruit roughly 
double the number of students for whom they will have internships. Several challenges arise from this 
strategy. For one, attrition rates vary each year. Consequently, Urban Alliance may end up with either too 
few or too many youth for the number of internships they have available. If too few youth, they risk having 
employers with vacancies; if too many, they risk not having internships for some youth. 

Perhaps more important is the possibility that the program ends up serving a motivated group of 
youth who may have done well in the absence of the Urban Alliance program. Although these youth can 
still benefit from the training and internships, if these youth were not truly at risk of disconnection, the 
resources may be better allocated to serve a more at-risk population of youth. However, those highly 
motivated youth who are involved in extracurricular activities such as sports or cheerleading cannot take 
advantage of the program, eliminating a group of youth who may benefit from the program or who may 
not need it. On the other hand, participants with initially lower motivation, often related to formidable 
challenges at home or in their schools, are not able to reap the rewards of the program. The results of the 
impact evaluation will be key for demonstrating the appropriateness of the self- selection aspect of Urban 
Alliance's program design. 

Finally, we note that Urban Alliance has developed internal evaluation capacity. Its full-time director 
of evaluation leads the organization's tracking of program alumni outcomes in education and 
employment. Ongoing evaluation is important for an organization to support continuous quality 
improvement. Willingness to devote resources to evaluation capacity, if used effectively, will pay off in 
program improvements which will lead to further program growth in the future. 

Conclusion 

Urban Alliance, nowproviding over 300 at-risk high school students in four metropolitan areas with job 
training, internships, mentorship, and help planning for college, is at a critical phase of organizational 
development. This baseline report and process study precedes two additional reports, altogether 
comprising the Urban Institute's evaluation of Urban Alliance's internship program. In this report, the 
authors reviewed existing analyses of youth interventions similar to Urban Alliance's internship program; 
explained the internship program model and its various components; described the population of youth 
taldngpart in this program; and presented findings from dozens of interviews and focus groups with 
program staff, youth, and other stakeholders. An interim report to be released in 2015 will include results 
of a follow-up survey that measures educational and employment outcomes for treatment and control 
group youth 9- 12 months after the program year concluded. A final report will also include results of a 
third survey administered 27- 30 months after the program year concluded. 

The Urban Alliance internship program has many promising components. The program design is 
unique, combining training in hard and soft skills during the pre- work and workshop environment, 
individual coaching and case management intended to help youth formulate and prepare for 
postsecondary goals, and a paid, professional internship. For each of these components, researchers 
found signs of success in observations and interviews, from well- crafted lesson plans that boosted 
students' skills in preparation for work, to frequent comments from youth and Program Coordinators 
lauding the strength of the relationships they established. Perhaps most notably, job mentors and 
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program staff dted considerable growth in youth's workplace and personal skills, especially their 
communication abilities and self-confidence; researchers, too, observed evidence of these improvements. 

Despite these promising signs of success, the organization continues to face challenges that it must 
overcome to better support youth. For example, significant attrition during pro- work and the internship 
phase means that the program misses serving significant segments of youth that could potentially benefit 
from the program. Program Coordinators reported feeling overwhelmed with their caseloads at times and 
conseguently lacked sufficient time for effective college planning with all students. The demanding nature 
of job site recruitment was a freguent complaint, but the Baltimore site in particular seemed to struggle in 
this effort. 

This study suggests that, despite such challenges. Urban Alliance is beneficial to its participants, at 
least in the short term. At the completion of the upcoming impact study, academics, practitioners, and 
policy leaders interested in similar interventions will know with greater certainty whether these benefits 
lead to longer-term gains. If so, the Urban Alliance model may set an important example for improving 
the lives of at- risk youth. 
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Appendix: Data and Methods 

What follows is a more detailed description of the program data, secondary data on youths' schools, 
neighborhoods, and academic performance, and gualitative data from interviews and focus groups that 
we collected during the course of the process study. 

Program Data 

Program data included the internship program application form, service delivery data, and organizational 
financial data. 

Application Form 

Urban Alliance high school internship program staff gave all applicants a 12-page application form to 
complete. The application reguested detailed contact information, demographics, GPA, attendance 
record, goals, career interests, work history, household structure, one teacher and one non- teacher 
recommendation, and parental consent forms. The application also serves as the baseline survey for the 
evaluation. 15 Urban Alliance provided the Urban Institute with the paper applications for all applicants. 
We entered a subset of fields relevant for the study into an electronic database. Unfortunately, item 
nonresponse was high for some fields in the application survey, making it impossible to reliably analyze 
household income, receipt of public benefits, education level of household members, and recommenders' 
assessments of youths' hard and soft skills. 

Service-Delivery Data 

In late 2011, Urban Alliance launched a new case management system using a Salesforce web- based 
framework. This system was designed to capture case management and program implementation 
information such as youth attendance in training workshops and internships, staff contact with youth, 
program completion, and progress in post- high school planning actions (e.g., submitting college 
applications, submitting a Free Application for Federal Student Aid [FAFSA], and producing an updated 
resume) . Urban Alliance provided this service delivery information to the Urban Institute and also 
provided Excel files with information about services provided before the introduction of the Salesforce 
system. J ob-site attendance records, whether extracted from the Salesforce system or predating it, proved 
to be unreliable. For more accurate measures of length of youths' internships and exact measures of their 
earnings, we used financial data provided to us by Urban Alliance that summarized hours worked and 
wages earned per two- week pay period. 

Program Costs 

To better understand the costs associated with maintaining the Urban Alliance internship program, we 
collected official audited financial information related to the internship programs in Washington and 
Baltimore for fiscal year 2012. We also obtained employee- level wage and time allocation information 
including employee title, site location, and time allotted to the internship program. We reclassified the 
audited financial information into broad categories of spending to assess how the various costs of the 
program— such as student wages, staff labor, and rent— compare to one another. Finally, we translated 
these costs to a per- student basis to derive a basic measure of the how cost intensive the program is per 
youth participant. 

Secondary Data 

Youth- and School-Level Education Data 

With the help of Urban Alliance, we obtained youth- level data for program applicants from Baltimore and 
Washington, DC, public schools and charter schools. The data used for this report includes GPAs, English 
language learner status, and special education status. 

We also pulled school- level performance data for each school for 2011 from Maryland and 
Washington, DC's boards of education. 16 To understand the relative performance of schools attended by 
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Urban Alliance youth, we gathered information not only on the schools they attended, but also on all 
schools in Maryland and Washington, DC. We ranked each school attended by program participants in 
Maryland and in DC according to their average 10th grade reading and math standardized test scores, 
determining each school's percentile rank among schools in that state/ district. 

Additionally, we used National Center for Education Statistics 2010 Common Core data 17 for school- 
level information about free and reduced- price lunch eligibility and racial composition. We linked the 
Common Core and performance data with Urban Alliance applicant records to better understand youths' 
educational environments, opportunities, and challenges. 

Neighborhood Data 

We used address information from program applications to geocode the locations where youth lived at the 
time of application. We matched their locations with census tract- level data from the US decennial census 
and the American Community Survey to gain additional information about the neighborhoods where the 
youth live, including poverty rate, unemployment rate, and racial and ethnic composition. 

Interviews and Focus Groups 

We conducted semi- structured interviews and focus groups with Urban Alliance staff, job mentors, school 
counselors, and youth participants. With the permission of respondents, we audio- recorded each 
interview and focus group and had those recordings transcribed. We coded these transcripts with key 
themes using qualitative analysis software. 

Interviews 

Urban Alliance staff interviewed four members of Urban Alliance's senior leadership, two city- specific 
program managers, and ten Program Coordinators. We conducted the interviews with program managers 
in summer 20 12 and with senior leadership staff in the spring of 20 13. Interviews of Program 
Coordinators occurred in summer 2012 and again in spring 2013 to incorporate the experiences and 
opinions of staff that had joined Urban Alliance after the first round of interviews, as well as to capture 
any changes in the program. In total, we were able to interview every staff person directly involved in 
administering the high school internship program. 

Topics for these interviews varied based on the respondent's role, but included typical work 
responsibilities and perspectives of each respondent, experiences in interacting with the youth, program 
goals, views on different program components, and recommendations for future change. Staff interviews 
lasted about 40 minutes each; all but one were conducted in person. 

In addition to the staff interviews, we conducted two interviews with school counselors to gain a 
school's perspective on the recruitment process and the value and challenges of the Urban Alliance 
program. Our questions focused on the mechanics of the recruitment process, how school counselors 
decide who to encourage to apply, the value of the program, and under what situations the program is 
appropriate for a student. School counselor interviews were conducted over the phone and lasted 
approximately 30 minutes. 

We conducted five interviews with job mentors to gain an employer perspective of the Urban Alliance 
program and its interns. Questions focused on job mentors' motivations for taking on an intern, what 
value they saw in the program for themselves and their organizations, and what value they saw in the 
program for the interns. J ob mentor interviews were conducted over the phone and lasted roughly 30 
minutes. 

Finally, we conducted six interviews with youth who started the Urban Alliance program but did not 
finish. While program completers were engaged as a part of focus groups, because program non- 
completers were difficult to convene, they were interviewed individually. Questions focused on why the 
youth applied for the program, their impressions of the program, and why they left the program. 
Interviews with these youth were conducted over the phone and were brief, typically 20 minutes. 
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Focus Groups 

We conducted four focus groups with youth participating in the Urban Alliance program during the 2 0 11- 
12 program year. These included one focus group each in Washington, DC, and Baltimore in the fall (at 
the end of pre- work training but before internships started), and one each in Washington, DC, and 
Baltimore during the following summer (when participants were on the verge of completing their 
internships and becoming program alumni) . Five to ten youth participated in each focus group. For the 
pre- work focus groups, youth were selected at random to participate. Where possible, the post- program 
focus groups included the same youth who participated in the pre- program focus groups. This overlap in 
participants allowed us to observe whether there were any apparent changes in individuals' perceptions 
between the beginning and end of the program. 

In the pre- work focus groups, discussions focused on youth motivations for applying for the program, 
their views on pre- work, expectations for their internships, and goals following high school graduation. In 
the end- of- program focus groups, discussions focused on experiences with their internships, views on the 
value of pre- work and workshops, the role of the Program Coordinators in their experience, and plans for 
the future. Focus groups lasted 40 to 50 minutes. 

Observations 

Over two years, we observed training sessions for youth in Washington, DC, and Baltimore. These 
included eight pre- work training sessions and four Friday workshop training sessions. We recorded notes 
on multiple aspects of each training session, including the resources used or needed for each session; 
student engagement; teaching technigues used; staff preparedness, tone, and approach; and material 
covered and its applicability to Urban Alliance outcome goals. We also observed one orientation session 
for new and returning job mentors in fall 2011. For this session, we observed job mentor attendance and 
attitudes, session content and emphasis, and participant guestions. 

We wrote and reviewed detailed notes from these site visits, which contributed to our understanding 
of the pre- work training and Friday workshops, as well as Urban Alliance staff responsibilities and youth 
participation in, and attitudes toward, training. 

Notes 

1. US Census Bureau, 20 12 American Community Survey, one- year estimates. 

2. The estimate for DC comes from "Free and Reduced- Price Lunch Eligibility," Kids Count Data Center, acoessed 
June 11, 2014, http://datacenter.kidscount.org/data/tables/4778-free-and-reduced-price-lunch- 
eligibility?loc=10&loct=3#detailed/ 3/ any/ false/ 1024,937,809,712,517/ 3534,894,897/ 11147. The estimate for 
Baltimore comes from "Students Receiving Free and Reduced School Meals," Kids Count Data Center, accessed J une 
11 20 14, http:// datacenter.kidscount.org/ data/ tables/ 70 78-students- receiving- free- and- reduced- school- 
meals?loc=2 2 &locl -3#det.ailecl/ 3/ 106/ false/ 36,868,867, 133,38/ any/ 14091, 14092. 

3. Estimates are for the graduating dass of 20 13. Baltimore estimates from 20 13 Maryland Report Card, "Baltimore 
City Graduation Rate: Four- year Adjusted Cohort," Maryland State Department of Education, accessed May 20, 2014, 
http:// www.mdreportcard.org/ CohortGradRate.aspx?PV=160: 12:30 :XXXX: 1:N:0: 13: 1: 1:0: 1: 1: 1:3. DC estimates from 
"DC 20 13 Adjusted Cohort Graduation Rate," District of Columbia Office of the State Superintendent of Education, 
accessed May 20, 20 14, 

http:/ / osse.dc.gov/ sites/ default/ files/ dc/ sites/ osse/ publication/ attachments/ DC%2020 13%20ADJ USTED%20COH 
ORT%2 0 GRADUAIT ON%2 0 RATE%2 0 state%2 0 summary_ 0 .pdf. 

4. The GED consists of four tests that certify passers' high school- level academic skills. 

5. There are exceptions; some programs appear to be targeted toward (or more attractive to) males. For example, the 
National Guard Youth ChalleNGe Program is 88 percent male (Millenky et al. 20 11), and J ob Corps is 59 percent male 
(Schochet et al. 2006). 

6. Of the 965 youth for whom data is available on GPA as of junior year, the data was from school records for 657 
youth (68 percent). It was reported by a counselor for 253 youth (26 percent) and by the youth for 55 (6 percent). 
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7. In recent yeans, many Baltimore and Washington, DC schools have become certified to offer free lunch to all 
students. Therefore, eligibility of individual students is no longer determined; the statistics shown here give an 
estimate of what eligibility would be in these schools if it was still determined at the student level. In the 20 12- 13 
school year, students from a four- member household with income below $42,643 qualified for a reduced- price meal, 
according to federal guidelines. 

8 . In the 20 11- 12 and 20 12- 13 program yeans, thene was no online application; most youth filled out applications by 
hand and the applications wene faxed on mailed to Unban Alliance. Fon the 20 13- 14 program year. Urban Alliance has 
provided an online application. 

9. For example, on several occasions, we heard fairly loud drumming coming from a nearby room. 

10 . In this context, "code- switching" refers to switching language style from vernacular English to standard American 
English (or vice versa) as youth transition in and out of formal and informal settings. 

11. Urban Alliance does not keep systematic information on the number of years each job site participates in the 
program. 

12. There is modest variation between the two sites. In Baltimore, 57 percent of applicants completed pre-work 
training versus 51 percent in Washington, DC. 

13. We define completion as remaining involved in the internship and workshops as of J une; a portion of participants 
are unable to continue through the summer due to other commitments. 

14. This indudes community colleges, where positive returns to credits earned have been demonstrated (Kane and 
Rouse 1999). 

15. Participation in the evaluation was voluntary; dedining to partidpate in the study had no bearing on partidpation 
in the Urban Alliance program. In practice, no youth who applied to the program declined to partidpate in the 
evaluation study. 

16. For Baltimore, we used the "20 11 HSA English Data" and "20 11 HSA Algebra Data" data files from "Data 
Downloads," 20 13 Maryland State Report Card, accessed J une 17th, 20 13, 

http://msp.msde.state.md.us/ downloadindex.aspx?K=99AAAA. For DC, we used data reports for each school from 
20 11, accessed through "Assessment and Accountability in the Distrid of Columbia," District of Columbia Office of the 
State Superintendent of Education, accessed onj une 17, 2013, http://ndb.osse.dc.gov/reportcards.asp. 

17. National Center for Education Statistics, "Public Elementary/ Secondary School Universe Survey Data, 2009- 10," 
accessed J une 6, 2013, http://nces.ed.gov/ccd/pubschuniv.asp. 
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